[Openmcl-devel] Checking ivector space use

Jon S. Anthony j-anthony at comcast.net
Mon Mar 15 09:10:20 PDT 2010


On Mon, 2010-03-15 at 11:40 -0400, Gail Zacharias wrote:
> At 3/15/2010 11:13 AM, Jon S. Anthony wrote:
> >Hi,
> >
> >Is there any "standard" way to check up on ivector (which I suppose
> >really means foreign memory) use?  I tried room, heap-utilization, and
> >time, and none report anything about this.  Since the GC pays no
> >attention to this stuff, that makes some sense.  But maybe there's
> >something I'm missing.
> 
> Terminology problem here - ivectors are things like strings and bit 
> vectors.  You must mean something else...

Well, could be.  I specifically I am looking at ivectors, as made by
make-heap-ivector.  The documentation for that says that these are
allocated in "foreign memory" and that the GC pays no attention to these
objects.

So, for example, given:

(defun make-page-vector (size)
  #+ccl
  (multiple-value-bind (vector pointer)
      (ccl:make-heap-ivector size '(unsigned-byte 8))
    (list pointer vector)))

And then,

? (time (let ((mem))
  (dotimes (i 20)
    (push (make-page-vector (* 8 (* 1024 1024))) mem))
  mem))
(LET ((MEM)) (DOTIMES (I 20) (PUSH (MAKE-PAGE-VECTOR (* 8 (* 1024
1024))) MEM)) MEM) took 0 milliseconds (0.000 seconds) to run 
                    with 2 available CPU cores.
During that period, 0 milliseconds (0.000 seconds) were spent in user
mode
                    0 milliseconds (0.000 seconds) were spent in system
mode
 2,080 bytes of memory allocated.
 20 minor page faults, 0 major page faults, 0 swaps.
((#<A Foreign Pointer #xB47C7014> #<VECTOR 8388608 type (UNSIGNED-BYTE
8), simple>) (#<A Foreign Pointer #xB4FC8014> #<VECTOR 8388608 type
(UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer #xB57C9014> #<VECTOR
8388608 type (UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer
#xB5FCA014> #<VECTOR 8388608 type (UNSIGNED-BYTE 8), simple>) (#<A
Foreign Pointer #xB67CB014> #<VECTOR 8388608 type (UNSIGNED-BYTE 8),
simple>) (#<A Foreign Pointer #xB6FCC014> #<VECTOR 8388608 type
(UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer #xB77CD014> #<VECTOR
8388608 type (UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer #x7419014>
#<VECTOR 8388608 type (UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer
#x6C18014> #<VECTOR 8388608 type (UNSIGNED-BYTE 8), simple>) (#<A
Foreign Pointer #x6417014> #<VECTOR 8388608 type (UNSIGNED-BYTE 8),
simple>) (#<A Foreign Pointer #x4F5B014> #<VECTOR 8388608 type
(UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer #x475A014> #<VECTOR
8388608 type (UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer #x3F59014>
#<VECTOR 8388608 type (UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer
#x3758014> #<VECTOR 8388608 type (UNSIGNED-BYTE 8), simple>) (#<A
Foreign Pointer #x2F57014> #<VECTOR 8388608 type (UNSIGNED-BYTE 8),
simple>) (#<A Foreign Pointer #x2756014> #<VECTOR 8388608 type
(UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer #x1F55014> #<VECTOR
8388608 type (UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer #x1754014>
#<VECTOR 8388608 type (UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer
#x5C16014> #<VECTOR 8388608 type (UNSIGNED-BYTE 8), simple>) (#<A
Foreign Pointer #xF53014> #<VECTOR 8388608 type (UNSIGNED-BYTE 8),
simple>))

The vectors are all allocated and so 160MB are allocated, which are not
reflected by the "2,080 bytes of memory allocated".


And

? (room t)
Approximately 16,646,144 bytes of memory can be allocated 
before the next full GC is triggered. 

                   Total Size             Free                 Used
Lisp Heap:       22085632 (21568K)   16646144 (16256K)    5439488
(5312K)
Stacks:           6211920 (6066K)     6209508 (6064K)        2412 (2K)
Static:          10968968 (10712K)          0 (0K)       10968968
(10712K)
2082.937 MB reserved for heap expansion.

Initial(0)
  cstack:      1386832 (1354K)      1386832 (1354K)            0 (0K)
  vstack:      1359872 (1328K)      1359760 (1328K)          112 (0K)
  tstack:       327680 (320K)        327368 (320K)           312 (0K)
listener(1)
  cstack:      1392640 (1360K)      1392640 (1360K)            0 (0K)
  vstack:      1155072 (1128K)      1154304 (1127K)          768 (1K)
  tstack:       589824 (576K)        588560 (575K)          1264 (1K)
NIL

I can't see the 160MB reflected here either.


And

? (heap-utilization)
Object type                       Count     Logical size   Physical size
% of Heap
                                             (in bytes)      (in bytes)
FUNCTION                          12760         5675756         5752600
35.06%
SIMPLE-BASE-STRING                51374         3155740         3458928
21.08%
SIMPLE-VECTOR                     25117         1243176         1397288
8.52%
SYMBOL                            41001         1148028         1312032
8.00%
CONS                             162288         1298304         1298304
7.91%
HASH-TABLE-VECTOR                    81         1064944         1065592
6.49%
SIMPLE-UNSIGNED-WORD-VECTOR        5051          691670          732080
4.46%
STRUCTURE                         15737          323328          444632
2.71%
SIMPLE-SIGNED-WORD-VECTOR             2          266480          266496
1.62%
SLOT-VECTOR                        4922          212244          243616
1.48%
SIMPLE-UNSIGNED-BYTE-VECTOR        4030          138067          161664
0.99%
INTERNAL-STRUCTURE                 2689           70176           81976
0.50%
STANDARD-INSTANCE                  4459           53508           71344
0.43%
BIGNUM                             5026           29744           56112
0.34%
SIMPLE-BIT-VECTOR                   545           44823           48856
0.30%
SIMPLE-UNSIGNED-LONG-VECTOR           4            4164            4192
0.03%
LOCK                                129            3096            4128
0.03%
MACPTR                              178            3176            3888
0.02%
SHORT-FLOAT                         163             652            1304
0.01%
DOUBLE-FLOAT                         73             876            1168
0.01%
BASIC-STREAM                         36             696             944
0.01%
PACKAGE                              15             480             600
0.00%
POPULATION                           22             268             360
0.00%
POOL                                 23              92             184
0.00%
VALUE-CELL                           19              76             152
0.00%
COMPLEX                               1               8              16
0.00%
Total                            335745        15429572        16408456
100.00%
? 

Not here either (doesn't look like ivector things are a kind of thing
reported on).  The number of MACPTRs increases from 158 (before the time
run) to 178 after, as expected.


/Jon





More information about the Openmcl-devel mailing list