[Openmcl-devel] Checking ivector space use
Gail Zacharias
gz at clozure.com
Mon Mar 15 10:02:26 PDT 2010
At 3/15/2010 12:10 PM, Jon S. Anthony wrote:
>On Mon, 2010-03-15 at 11:40 -0400, Gail Zacharias wrote:
> > At 3/15/2010 11:13 AM, Jon S. Anthony wrote:
> > >Hi,
> > >
> > >Is there any "standard" way to check up on ivector (which I suppose
> > >really means foreign memory) use? I tried room, heap-utilization, and
> > >time, and none report anything about this. Since the GC pays no
> > >attention to this stuff, that makes some sense. But maybe there's
> > >something I'm missing.
> >
> > Terminology problem here - ivectors are things like strings and bit
> > vectors. You must mean something else...
>
>Well, could be. I specifically I am looking at ivectors, as made by
>make-heap-ivector. The documentation for that says that these are
>allocated in "foreign memory" and that the GC pays no attention to these
>objects.
Ah, ok. The "foreign memory" is just the malloc heap. You can get
info about the malloc heap with ccl::show-malloc-info:
CL-USER> (ccl::show-malloc-info)
arena size: 1056768/#x102000
number of unused chunks = 37
number of mmap'ed chunks = 0
total size of mmap'ed chunks = 0/#x0
total size of malloc'ed chunks = 810960/#xC5FD0
total size of free chunks = 245808/#x3C030
size of releaseable chunk = 134640/#x20DF0
>So, for example, given:
>
>(defun make-page-vector (size)
> #+ccl
> (multiple-value-bind (vector pointer)
> (ccl:make-heap-ivector size '(unsigned-byte 8))
> (list pointer vector)))
>
>And then,
>
>? (time (let ((mem))
> (dotimes (i 20)
> (push (make-page-vector (* 8 (* 1024 1024))) mem))
> mem))
>(LET ((MEM)) (DOTIMES (I 20) (PUSH (MAKE-PAGE-VECTOR (* 8 (* 1024
>1024))) MEM)) MEM) took 0 milliseconds (0.000 seconds) to run
> with 2 available CPU cores.
>During that period, 0 milliseconds (0.000 seconds) were spent in user
>mode
> 0 milliseconds (0.000 seconds) were spent in system
>mode
> 2,080 bytes of memory allocated.
> 20 minor page faults, 0 major page faults, 0 swaps.
>((#<A Foreign Pointer #xB47C7014> #<VECTOR 8388608 type (UNSIGNED-BYTE
>8), simple>) (#<A Foreign Pointer #xB4FC8014> #<VECTOR 8388608 type
>(UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer #xB57C9014> #<VECTOR
>8388608 type (UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer
>#xB5FCA014> #<VECTOR 8388608 type (UNSIGNED-BYTE 8), simple>) (#<A
>Foreign Pointer #xB67CB014> #<VECTOR 8388608 type (UNSIGNED-BYTE 8),
>simple>) (#<A Foreign Pointer #xB6FCC014> #<VECTOR 8388608 type
>(UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer #xB77CD014> #<VECTOR
>8388608 type (UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer #x7419014>
>#<VECTOR 8388608 type (UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer
>#x6C18014> #<VECTOR 8388608 type (UNSIGNED-BYTE 8), simple>) (#<A
>Foreign Pointer #x6417014> #<VECTOR 8388608 type (UNSIGNED-BYTE 8),
>simple>) (#<A Foreign Pointer #x4F5B014> #<VECTOR 8388608 type
>(UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer #x475A014> #<VECTOR
>8388608 type (UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer #x3F59014>
>#<VECTOR 8388608 type (UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer
>#x3758014> #<VECTOR 8388608 type (UNSIGNED-BYTE 8), simple>) (#<A
>Foreign Pointer #x2F57014> #<VECTOR 8388608 type (UNSIGNED-BYTE 8),
>simple>) (#<A Foreign Pointer #x2756014> #<VECTOR 8388608 type
>(UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer #x1F55014> #<VECTOR
>8388608 type (UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer #x1754014>
>#<VECTOR 8388608 type (UNSIGNED-BYTE 8), simple>) (#<A Foreign Pointer
>#x5C16014> #<VECTOR 8388608 type (UNSIGNED-BYTE 8), simple>) (#<A
>Foreign Pointer #xF53014> #<VECTOR 8388608 type (UNSIGNED-BYTE 8),
>simple>))
>
>The vectors are all allocated and so 160MB are allocated, which are not
>reflected by the "2,080 bytes of memory allocated".
>
>
>And
>
>? (room t)
>Approximately 16,646,144 bytes of memory can be allocated
>before the next full GC is triggered.
>
> Total Size Free Used
>Lisp Heap: 22085632 (21568K) 16646144 (16256K) 5439488
>(5312K)
>Stacks: 6211920 (6066K) 6209508 (6064K) 2412 (2K)
>Static: 10968968 (10712K) 0 (0K) 10968968
>(10712K)
>2082.937 MB reserved for heap expansion.
>
>Initial(0)
> cstack: 1386832 (1354K) 1386832 (1354K) 0 (0K)
> vstack: 1359872 (1328K) 1359760 (1328K) 112 (0K)
> tstack: 327680 (320K) 327368 (320K) 312 (0K)
>listener(1)
> cstack: 1392640 (1360K) 1392640 (1360K) 0 (0K)
> vstack: 1155072 (1128K) 1154304 (1127K) 768 (1K)
> tstack: 589824 (576K) 588560 (575K) 1264 (1K)
>NIL
>
>I can't see the 160MB reflected here either.
>
>
>And
>
>? (heap-utilization)
>Object type Count Logical size Physical size
>% of Heap
> (in bytes) (in bytes)
>FUNCTION 12760 5675756 5752600
>35.06%
>SIMPLE-BASE-STRING 51374 3155740 3458928
>21.08%
>SIMPLE-VECTOR 25117 1243176 1397288
>8.52%
>SYMBOL 41001 1148028 1312032
>8.00%
>CONS 162288 1298304 1298304
>7.91%
>HASH-TABLE-VECTOR 81 1064944 1065592
>6.49%
>SIMPLE-UNSIGNED-WORD-VECTOR 5051 691670 732080
>4.46%
>STRUCTURE 15737 323328 444632
>2.71%
>SIMPLE-SIGNED-WORD-VECTOR 2 266480 266496
>1.62%
>SLOT-VECTOR 4922 212244 243616
>1.48%
>SIMPLE-UNSIGNED-BYTE-VECTOR 4030 138067 161664
>0.99%
>INTERNAL-STRUCTURE 2689 70176 81976
>0.50%
>STANDARD-INSTANCE 4459 53508 71344
>0.43%
>BIGNUM 5026 29744 56112
>0.34%
>SIMPLE-BIT-VECTOR 545 44823 48856
>0.30%
>SIMPLE-UNSIGNED-LONG-VECTOR 4 4164 4192
>0.03%
>LOCK 129 3096 4128
>0.03%
>MACPTR 178 3176 3888
>0.02%
>SHORT-FLOAT 163 652 1304
>0.01%
>DOUBLE-FLOAT 73 876 1168
>0.01%
>BASIC-STREAM 36 696 944
>0.01%
>PACKAGE 15 480 600
>0.00%
>POPULATION 22 268 360
>0.00%
>POOL 23 92 184
>0.00%
>VALUE-CELL 19 76 152
>0.00%
>COMPLEX 1 8 16
>0.00%
>Total 335745 15429572 16408456
>100.00%
>?
>
>Not here either (doesn't look like ivector things are a kind of thing
>reported on). The number of MACPTRs increases from 158 (before the time
>run) to 178 after, as expected.
>
>
>/Jon
More information about the Openmcl-devel
mailing list