[Openmcl-devel] Checking ivector space use

Gail Zacharias gz at clozure.com
Mon Mar 15 17:02:26 UTC 2010


At 3/15/2010 12:10 PM, Jon S. Anthony wrote:
>On Mon, 2010-03-15 at 11:40 -0400, Gail Zacharias wrote:
> > At 3/15/2010 11:13 AM, Jon S. Anthony wrote:
> > >Hi,
> > >
> > >Is there any "standard" way to check up on ivector (which I suppose
> > >really means foreign memory) use?  I tried room, heap-utilization, and
> > >time, and none report anything about this.  Since the GC pays no
> > >attention to this stuff, that makes some sense.  But maybe there's
> > >something I'm missing.
> >
> > Terminology problem here - ivectors are things like strings and bit
> > vectors.  You must mean something else...
>
>Well, could be.  I specifically I am looking at ivectors, as made by
>make-heap-ivector.  The documentation for that says that these are
>allocated in "foreign memory" and that the GC pays no attention to these
>objects.

Ah, ok.  The "foreign memory" is just the malloc heap.  You can get 
info about the malloc heap with ccl::show-malloc-info:

CL-USER> (ccl::show-malloc-info)
  arena size: 1056768/#x102000
  number of unused chunks = 37
  number of mmap'ed chunks = 0
  total size of mmap'ed chunks = 0/#x0
  total size of malloc'ed chunks = 810960/#xC5FD0
  total size of free chunks = 245808/#x3C030
  size of releaseable chunk = 134640/#x20DF0



>So, for example, given:
>
>(defun make-page-vector (size)
>   #+ccl
>   (multiple-value-bind (vector pointer)
>       (ccl:make-heap-ivector size '(unsigned-byte 8))
>     (list pointer vector)))
>
>And then,
>
>? (time (let ((mem))
>   (dotimes (i 20)
>     (push (make-page-vector (* 8 (* 1024 1024))) mem))
>   mem))
>(LET ((MEM)) (DOTIMES (I 20) (PUSH (MAKE-PAGE-VECTOR (* 8 (* 1024
>1024))) MEM)) MEM) took 0 milliseconds (0.000 seconds) to run
>                     with 2 available CPU cores.
>During that period, 0 milliseconds (0.000 seconds) were spent in user
>mode
>                     0 milliseconds (0.000 seconds) were spent in system
>mode
>  2,080 bytes of memory allocated.
>  20 minor page faults, 0 major page faults, 0 swaps.
>((#<A Foreign Pointer #xB47C7014> #<VECTOR 8388608 type (UNSIGNED-BYTE
>8), simple>) (#<A Foreign Pointer #xB4FC8014> #<VECTOR 8388608 type
>(UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer #xB57C9014> #<VECTOR
>8388608 type (UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer
>#xB5FCA014> #<VECTOR 8388608 type (UNSIGNED-BYTE 8), simple>) (#<A
>Foreign Pointer #xB67CB014> #<VECTOR 8388608 type (UNSIGNED-BYTE 8),
>simple>) (#<A Foreign Pointer #xB6FCC014> #<VECTOR 8388608 type
>(UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer #xB77CD014> #<VECTOR
>8388608 type (UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer #x7419014>
>#<VECTOR 8388608 type (UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer
>#x6C18014> #<VECTOR 8388608 type (UNSIGNED-BYTE 8), simple>) (#<A
>Foreign Pointer #x6417014> #<VECTOR 8388608 type (UNSIGNED-BYTE 8),
>simple>) (#<A Foreign Pointer #x4F5B014> #<VECTOR 8388608 type
>(UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer #x475A014> #<VECTOR
>8388608 type (UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer #x3F59014>
>#<VECTOR 8388608 type (UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer
>#x3758014> #<VECTOR 8388608 type (UNSIGNED-BYTE 8), simple>) (#<A
>Foreign Pointer #x2F57014> #<VECTOR 8388608 type (UNSIGNED-BYTE 8),
>simple>) (#<A Foreign Pointer #x2756014> #<VECTOR 8388608 type
>(UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer #x1F55014> #<VECTOR
>8388608 type (UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer #x1754014>
>#<VECTOR 8388608 type (UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer
>#x5C16014> #<VECTOR 8388608 type (UNSIGNED-BYTE 8), simple>) (#<A
>Foreign Pointer #xF53014> #<VECTOR 8388608 type (UNSIGNED-BYTE 8),
>simple>))
>
>The vectors are all allocated and so 160MB are allocated, which are not
>reflected by the "2,080 bytes of memory allocated".
>
>
>And
>
>? (room t)
>Approximately 16,646,144 bytes of memory can be allocated
>before the next full GC is triggered.
>
>                    Total Size             Free                 Used
>Lisp Heap:       22085632 (21568K)   16646144 (16256K)    5439488
>(5312K)
>Stacks:           6211920 (6066K)     6209508 (6064K)        2412 (2K)
>Static:          10968968 (10712K)          0 (0K)       10968968
>(10712K)
>2082.937 MB reserved for heap expansion.
>
>Initial(0)
>   cstack:      1386832 (1354K)      1386832 (1354K)            0 (0K)
>   vstack:      1359872 (1328K)      1359760 (1328K)          112 (0K)
>   tstack:       327680 (320K)        327368 (320K)           312 (0K)
>listener(1)
>   cstack:      1392640 (1360K)      1392640 (1360K)            0 (0K)
>   vstack:      1155072 (1128K)      1154304 (1127K)          768 (1K)
>   tstack:       589824 (576K)        588560 (575K)          1264 (1K)
>NIL
>
>I can't see the 160MB reflected here either.
>
>
>And
>
>? (heap-utilization)
>Object type                       Count     Logical size   Physical size
>% of Heap
>                                              (in bytes)      (in bytes)
>FUNCTION                          12760         5675756         5752600
>35.06%
>SIMPLE-BASE-STRING                51374         3155740         3458928
>21.08%
>SIMPLE-VECTOR                     25117         1243176         1397288
>8.52%
>SYMBOL                            41001         1148028         1312032
>8.00%
>CONS                             162288         1298304         1298304
>7.91%
>HASH-TABLE-VECTOR                    81         1064944         1065592
>6.49%
>SIMPLE-UNSIGNED-WORD-VECTOR        5051          691670          732080
>4.46%
>STRUCTURE                         15737          323328          444632
>2.71%
>SIMPLE-SIGNED-WORD-VECTOR             2          266480          266496
>1.62%
>SLOT-VECTOR                        4922          212244          243616
>1.48%
>SIMPLE-UNSIGNED-BYTE-VECTOR        4030          138067          161664
>0.99%
>INTERNAL-STRUCTURE                 2689           70176           81976
>0.50%
>STANDARD-INSTANCE                  4459           53508           71344
>0.43%
>BIGNUM                             5026           29744           56112
>0.34%
>SIMPLE-BIT-VECTOR                   545           44823           48856
>0.30%
>SIMPLE-UNSIGNED-LONG-VECTOR           4            4164            4192
>0.03%
>LOCK                                129            3096            4128
>0.03%
>MACPTR                              178            3176            3888
>0.02%
>SHORT-FLOAT                         163             652            1304
>0.01%
>DOUBLE-FLOAT                         73             876            1168
>0.01%
>BASIC-STREAM                         36             696             944
>0.01%
>PACKAGE                              15             480             600
>0.00%
>POPULATION                           22             268             360
>0.00%
>POOL                                 23              92             184
>0.00%
>VALUE-CELL                           19              76             152
>0.00%
>COMPLEX                               1               8              16
>0.00%
>Total                            335745        15429572        16408456
>100.00%
>?
>
>Not here either (doesn't look like ivector things are a kind of thing
>reported on).  The number of MACPTRs increases from 158 (before the time
>run) to 178 after, as expected.
>
>
>/Jon




More information about the Openmcl-devel mailing list