[Openmcl-devel] native threads

[Shannon Spires]

Threads in CCL since version 0.14 (which was a long time ago) are native 
and thus preemptively scheduled by the operating system. Unlike in 
Allegro CL and Lispworks, SMP cannot easily be turned off in CCL: It's 
always on. And there is no "CCL scheduler".

-SS

On 1/5/24 9:58 AM, Tim McNerney wrote:
> Your "thread pools" approach sounds wise. While I don't (but should) 
> know what CCL does right now w.r.t. threads, the "modern thing to do" 
> is for Lisps to use native OS threads, which I can easily imagine take 
> some time to launch, but then take good advantage of symmetric 
> multiprocessing (SMP).  That's what Franz does anyway.  I assume, 
> based on the compiler output I've studied, that CCL supports SMP 
> threads. I would not expect that CCL runs its own scheduler anymore. 
> [Wizards: please correct me if I'm wrong].
>
> --Tim
>
> On 1/5/24 10:54 AM, David McClain wrote:
>> I have spent more than a decade implementing various versions of 
>> Actors systems. Some were along the lines you describe, launching 
>> native threads to perform the work of Actors.
>>
>> I finally concluded that, having just run a good race, you ought not 
>> kill the race horses, but rather retire them to the stable for 
>> another race. IOW, I now use thread pools, permanently launched and 
>> waiting for dispatch to another task. Thread startup entails a huge 
>> amount of overhead - allocating stack space and CPU registers, 
>> registering with OS, etc.
>>
>> - DM
>>
>>> On Jan 5, 2024, at 07:32, Taoufik Dachraoui 
>>> <dachraoui.taoufik at gmail.com> wrote:
>>>
>>> why ccl is much slower?
>>>
>>> taoufik at Ankbot:~/workspace/ccl/actor$ sbcl
>>> This is SBCL 2.3.11, an implementation of ANSI Common Lisp.
>>> More information about SBCL is available at <http://www.sbcl.org/>.
>>>
>>> SBCL is free software, provided as is, with absolutely no warranty.
>>> It is mostly in the public domain; some portions are provided under
>>> BSD-style licenses.  See the CREDITS and COPYING files in the
>>> distribution for more information.
>>> * (time (prog1 t (loop for i from 0 upto 100000000 collect i)))
>>> Evaluation took:
>>>   1.379 seconds of real time
>>>   1.379015 seconds of total run time (0.856185 user, 0.522830 system)
>>>   [ Real times consist of 1.143 seconds GC time, and 0.236 seconds 
>>> non-GC time. ]
>>>   [ Run times consist of 1.146 seconds GC time, and 0.234 seconds 
>>> non-GC time. ]
>>>   100.00% CPU
>>>   2,913,031,106 processor cycles
>>>   1,600,427,200 bytes consed
>>>
>>> T
>>> * (quit)
>>> taoufik at Ankbot:~/workspace/ccl/actor$ ccl
>>> Clozure Common Lisp Version 1.12.1 (v1.12.1-22-g6b1f1d3a) LinuxX8664
>>>
>>> For more information about CCL, please see http://ccl.clozure.com.
>>>
>>> CCL is free software.  It is distributed under the terms of the Apache
>>> Licence, Version 2.0.
>>> ? (time (prog1 t (loop for i from 0 upto 100000000 collect i)))
>>> (PROG1 T (LOOP FOR I FROM 0 UPTO 100000000 COLLECT I))
>>> took 11,946,514 microseconds (11.946514 seconds) to run.
>>>      11,147,722 microseconds (11.147722 seconds, 93.31%) of which 
>>> was spent in GC.
>>> During that period, and with 24 available CPU cores,
>>>      11,632,370 microseconds (11.632370 seconds) were spent in user mode
>>>         292,428 microseconds ( 0.292428 seconds) were spent in 
>>> system mode
>>>  1,600,000,032 bytes of memory allocated.
>>>  397,400 minor page faults, 0 major page faults, 0 swaps.
>>> T
>>> ?
>>>
>>> On Fri, Jan 5, 2024 at 3:28 PM Taoufik Dachraoui 
>>> <dachraoui.taoufik at gmail.com> wrote:
>>>
>>>     It looks like it is not because of the threads:
>>>
>>>     taoufik at Ankbot:~/workspace/ccl/actor$ sbcl
>>>     This is SBCL 2.3.11, an implementation of ANSI Common Lisp.
>>>     More information about SBCL is available at <http://www.sbcl.org/>.
>>>
>>>     SBCL is free software, provided as is, with absolutely no warranty.
>>>     It is mostly in the public domain; some portions are provided under
>>>     BSD-style licenses.  See the CREDITS and COPYING files in the
>>>     distribution for more information.
>>>     * (time (reduce #'+ (loop for i from 0 upto 100000000 collect i)))
>>>     Evaluation took:
>>>       1.836 seconds of real time
>>>       1.835223 seconds of total run time (1.351394 user, 0.483829
>>>     system)
>>>       [ Real times consist of 1.184 seconds GC time, and 0.652
>>>     seconds non-GC time. ]
>>>       [ Run times consist of 1.184 seconds GC time, and 0.652
>>>     seconds non-GC time. ]
>>>       99.95% CPU
>>>       3,876,850,246 processor cycles
>>>       1,600,427,200 bytes consed
>>>
>>>     5000000050000000
>>>     * taoufik at Ankbot:~/workspace/ccl/actor$ ccl
>>>     Clozure Common Lisp Version 1.12.1 (v1.12.1-22-g6b1f1d3a) LinuxX8664
>>>
>>>     For more information about CCL, please see http://ccl.clozure.com.
>>>
>>>     CCL is free software.  It is distributed under the terms of the
>>>     Apache
>>>     Licence, Version 2.0.
>>>     ? (time (reduce #'+ (loop for i from 0 upto 100000000 collect i)))
>>>     (REDUCE #'+ (LOOP FOR I FROM 0 UPTO 100000000 COLLECT I))
>>>     took 13,091,079 microseconds (13.091079 seconds) to run.
>>>          11,036,666 microseconds (11.036666 seconds, 84.31%) of
>>>     which was spent in GC.
>>>     During that period, and with 24 available CPU cores,
>>>          12,734,283 microseconds (12.734283 seconds) were spent in
>>>     user mode
>>>             336,005 microseconds ( 0.336005 seconds) were spent in
>>>     system mode
>>>      1,600,000,032 bytes of memory allocated.
>>>      397,400 minor page faults, 0 major page faults, 0 swaps.
>>>     5000000050000000
>>>     ?
>>>
>>>     On Fri, Jan 5, 2024 at 3:10 PM Taoufik Dachraoui
>>>     <dachraoui.taoufik at gmail.com> wrote:
>>>
>>>         Hi
>>>
>>>         In my current implementation of a classical actor model
>>>         I found that using ccl processes is much slower than sbcl
>>>         threads
>>>
>>>         to create a thread I use ccl:process-run-function, is there
>>>         another way to
>>>         create native threads that are much faster; I do not need
>>>         the ccl scheduling,
>>>         I want to create threads that are scheduled by the OS, I
>>>         think that the ccl
>>>         scheduler is the reason why my ccl tests are much slower
>>>         than the tests run
>>>         with sbcl
>>>
>>>         Regards
>>>         -- 
>>>         Taoufik Dachraoui
>>>
>>>
>>>
>>>     -- 
>>>     Taoufik Dachraoui
>>>
>>>
>>>
>>> -- 
>>> Taoufik Dachraoui
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clozure.com/pipermail/openmcl-devel/attachments/20240105/ec93fd18/attachment-0001.htm>