[Openmcl-devel] Binary IO...

[Jon S. Anthony]

Hi,

As part of a porting job for my graph store, I'm experimenting with
various binary IO variations (which need random access as well).
Originally, this was done in C.  I suppose I can still do that, but the
CCL version is so much better than the ACL variant that getting rid of
the C for it seems like a good - simpler - idea.  There are some other
possibilities as well but that is irrelevant here.  But, there is still
a head scratching aspect to the CL variant.

Enclosed are two simple programs, one in C and one in CL.  They both do
the same thing and the CL pretty much mimics the "C level" form of the
C.  All they do is write and read a binary file in 8MB chunks (320MB
worth).

Setting aside the "elapsed time" aspects (which seem to pretty clearly
be tied to the OS disk caching behavior) they are both pretty fast.  But
the C is still around 3X faster in general.  This seems to be due to the
fact that the CL burns up typically ~1.5 seconds in user mode.  The C
version typically runs with 0 (!) user mode time.  The system level time
of both is about the same ~800ms or so.

I would have thought that read-sequence and write-sequence basically map
to fread and fwrite respectively - especially (as in this case) the
sequence involved is a simple array of unsigned byte 8.

Long winded prelude to what would the CL variant be doing in user mode
(read/write-sequence) that the C version is able to (or "dangerously")
skips?  Off hand, it doesn't seem like the type determination/resolution
in read/write-sequence would burn up this much time.

C is compiled simply with gcc.  Here are a couple example timings:

C: gcc (GCC) 4.1.0 20060304 (x8632)

$ time ../megabyte-binary-io

0.004u 0.840s 0:00.86 97.6%     0+0k 0+0io 0pf+0w

$ time ../megabyte-binary-io

0.000u 0.736s 0:00.73 100.0%    0+0k 0+0io 0pf+0w


CL: (ccl-1.3 x8632)

? (time (main))
(MAIN) took 2,661 milliseconds (2.661 seconds) to run 
                    with 2 available CPU cores.
During that period, 1,680 milliseconds (1.680 seconds) were spent in
user mode
                    904 milliseconds (0.904 seconds) were spent in
system mode
6 milliseconds (0.006 seconds) was spent in GC.
 8,394,104 bytes of memory allocated.
NIL
? (time (main))
(MAIN) took 7,590 milliseconds (7.590 seconds) to run 
                    with 2 available CPU cores.
During that period, 1,716 milliseconds (1.716 seconds) were spent in
user mode
                    768 milliseconds (0.768 seconds) were spent in
system mode
6 milliseconds (0.006 seconds) was spent in GC.
 8,394,104 bytes of memory allocated.


Thanks,

/Jon


-------------- next part --------------
A non-text attachment was scrubbed...
Name: megabyte-binary-io.lisp
Type: text/x-emacs-lisp
Size: 1399 bytes
Desc: not available
URL: <https://lists.clozure.com/pipermail/openmcl-devel/attachments/20090607/7a72343e/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: megabyte-binary-io.c
Type: text/x-csrc
Size: 717 bytes
Desc: not available
URL: <https://lists.clozure.com/pipermail/openmcl-devel/attachments/20090607/7a72343e/attachment-0001.bin>