[Openmcl-devel] native threads

[Shannon Spires]

This fixes the problem, at least on CCL on MacOS (my Linux box is not 
handy ATM):

*(set-lisp-heap-gc-threshold 671350784) *; default is 33554432
(use-lisp-heap-gc-threshold)
(gc) ; clear memory as much as possible

(defun foo1 ()
   (prog1 t (loop for i from 0 upto 100000000 collect i)))

(time (foo1))
(FOO1)
took 3,604,166 microseconds (3.604166 seconds) to run.
      2,193,000 microseconds (2.193000 seconds, 60.85%) of which was 
spent in GC.
During that period, and with 16 available CPU cores,
      3,130,749 microseconds (3.130749 seconds) were spent in user mode
        501,264 microseconds (0.501264 seconds) were spent in system mode
  1,600,000,032 bytes of memory allocated.
  227,013 minor page faults, 0 major page faults, 0 swaps.
T

As others have pointed out, there's a lot of GC going on here.

The garbage collector in CCL is tuned not to hog system memory 
unnecessarily, so every time the GC runs, it will allocate more memory 
from the system if it needs it. By calling#'set-lisp-heap-gc-threshold 
with a larger number than the default, we're essentially telling CCL to 
allocate larger chunks of memory from the system. This means the GC has 
to run less frequently, which makes the above code run much faster.

In order to run the benchmark with SBCL, I had to use the argument 
"--dynamic-space-size 4000000" just to prevent the heap from being 
exhausted.

As others have also pointed out, this is not a particularly realistic 
benchmark. Practical CL implementations are not by default tuned to 
allow huge amounts of worthless garbage to be created quickly. But 
tuning them to allow it is possible.

-SS

On 1/5/24 7:32 AM, Taoufik Dachraoui wrote:
> why ccl is much slower?
>
> taoufik at Ankbot:~/workspace/ccl/actor$ sbcl
> This is SBCL 2.3.11, an implementation of ANSI Common Lisp.
> More information about SBCL is available at <http://www.sbcl.org/>.
>
> SBCL is free software, provided as is, with absolutely no warranty.
> It is mostly in the public domain; some portions are provided under
> BSD-style licenses.  See the CREDITS and COPYING files in the
> distribution for more information.
> * (time (prog1 t (loop for i from 0 upto 100000000 collect i)))
> Evaluation took:
>   1.379 seconds of real time
>   1.379015 seconds of total run time (0.856185 user, 0.522830 system)
>   [ Real times consist of 1.143 seconds GC time, and 0.236 seconds 
> non-GC time. ]
>   [ Run times consist of 1.146 seconds GC time, and 0.234 seconds 
> non-GC time. ]
>   100.00% CPU
>   2,913,031,106 processor cycles
>   1,600,427,200 bytes consed
>
> T
> * (quit)
> taoufik at Ankbot:~/workspace/ccl/actor$ ccl
> Clozure Common Lisp Version 1.12.1 (v1.12.1-22-g6b1f1d3a) LinuxX8664
>
> For more information about CCL, please see http://ccl.clozure.com.
>
> CCL is free software.  It is distributed under the terms of the Apache
> Licence, Version 2.0.
> ? (time (prog1 t (loop for i from 0 upto 100000000 collect i)))
> (PROG1 T (LOOP FOR I FROM 0 UPTO 100000000 COLLECT I))
> took 11,946,514 microseconds (11.946514 seconds) to run.
>      11,147,722 microseconds (11.147722 seconds, 93.31%) of which was 
> spent in GC.
> During that period, and with 24 available CPU cores,
>      11,632,370 microseconds (11.632370 seconds) were spent in user mode
>         292,428 microseconds ( 0.292428 seconds) were spent in system mode
>  1,600,000,032 bytes of memory allocated.
>  397,400 minor page faults, 0 major page faults, 0 swaps.
> T
> ?
>
> On Fri, Jan 5, 2024 at 3:28 PM Taoufik Dachraoui 
> <dachraoui.taoufik at gmail.com> wrote:
>
>     It looks like it is not because of the threads:
>
>     taoufik at Ankbot:~/workspace/ccl/actor$ sbcl
>     This is SBCL 2.3.11, an implementation of ANSI Common Lisp.
>     More information about SBCL is available at <http://www.sbcl.org/>.
>
>     SBCL is free software, provided as is, with absolutely no warranty.
>     It is mostly in the public domain; some portions are provided under
>     BSD-style licenses.  See the CREDITS and COPYING files in the
>     distribution for more information.
>     * (time (reduce #'+ (loop for i from 0 upto 100000000 collect i)))
>     Evaluation took:
>       1.836 seconds of real time
>       1.835223 seconds of total run time (1.351394 user, 0.483829 system)
>       [ Real times consist of 1.184 seconds GC time, and 0.652 seconds
>     non-GC time. ]
>       [ Run times consist of 1.184 seconds GC time, and 0.652 seconds
>     non-GC time. ]
>       99.95% CPU
>       3,876,850,246 processor cycles
>       1,600,427,200 bytes consed
>
>     5000000050000000
>     * taoufik at Ankbot:~/workspace/ccl/actor$ ccl
>     Clozure Common Lisp Version 1.12.1 (v1.12.1-22-g6b1f1d3a) LinuxX8664
>
>     For more information about CCL, please see http://ccl.clozure.com.
>
>     CCL is free software.  It is distributed under the terms of the Apache
>     Licence, Version 2.0.
>     ? (time (reduce #'+ (loop for i from 0 upto 100000000 collect i)))
>     (REDUCE #'+ (LOOP FOR I FROM 0 UPTO 100000000 COLLECT I))
>     took 13,091,079 microseconds (13.091079 seconds) to run.
>          11,036,666 microseconds (11.036666 seconds, 84.31%) of which
>     was spent in GC.
>     During that period, and with 24 available CPU cores,
>          12,734,283 microseconds (12.734283 seconds) were spent in
>     user mode
>             336,005 microseconds ( 0.336005 seconds) were spent in
>     system mode
>      1,600,000,032 bytes of memory allocated.
>      397,400 minor page faults, 0 major page faults, 0 swaps.
>     5000000050000000
>     ?
>
>     On Fri, Jan 5, 2024 at 3:10 PM Taoufik Dachraoui
>     <dachraoui.taoufik at gmail.com> wrote:
>
>         Hi
>
>         In my current implementation of a classical actor model
>         I found that using ccl processes is much slower than sbcl threads
>
>         to create a thread I use ccl:process-run-function, is there
>         another way to
>         create native threads that are much faster; I do not need the
>         ccl scheduling,
>         I want to create threads that are scheduled by the OS, I think
>         that the ccl
>         scheduler is the reason why my ccl tests are much slower than
>         the tests run
>         with sbcl
>
>         Regards
>         -- 
>         Taoufik Dachraoui
>
>
>
>     -- 
>     Taoufik Dachraoui
>
>
>
> -- 
> Taoufik Dachraoui
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clozure.com/pipermail/openmcl-devel/attachments/20240105/8987f696/attachment.htm>