[Openmcl-devel] Managing multi-threaded access to list

[Gary Byers]

On Sun, 28 May 2006, Joseph Oswald wrote:

>
>  Furthermore, is there any need to destroy semaphores when the
> connection goes away? Are they a limited OS resource?

No and yes.

The "no" part: semaphores in OpenMCL are first-class lisp objects that
have "indefinite extent": they exist until the GC can prove that they
can no longer be referenced (the same, in general, as CONS cells and
STRINGs and ...).  There's no way to explicitly deallocate one (just
like CONS cells and STRINGs and ...).  When the GC notices that a
semaphore has become garbage, it frees the OS-level resources
associated with the lisp semaphore object as well as freeing the lisp
object.

The "yes" part: it's hard to quantify exactly, but semaphores do
involve OS kernel resources and those resources are finite. I don't
know exactly what the upper bound on the number of (Mach) semaphores
that a process can create is on OSX, but I think that this may depend
on a lot of factors - OS load, physical memory, etc. - and it may be
the case (is that hand-wavy enough ?) that things other things sometimes
fail, bog down, and/or misbehave before MAKE-SEMAPHORE fails outright.

I just tried:

(defun foo (n)
   "push newly-created semaphores on a list to keep them from getting GCed"
   (let* ((l ()))
     (dotimes (i n)
       (push (make-semaphore) l))))

(foo 10000)  ran pretty quicky, and a subsequent GC was only a little
slower than normal.

(foo 100000) ran more than much more than 10X more slowly than that, and
a subsequent GC:


? (time (gc))
(GC) took 2,199 milliseconds (2.199 seconds) to run.
Of that, 288 milliseconds (0.288 seconds) were spent in user mode
          1,911 milliseconds (1.911 seconds) were spent in system mode
2,199 milliseconds (2.199 seconds) was spent in GC.

spent almost 2 seconds in the OS, freeing all of those semaphores
(and a fairly long time in the GC proving that the 100000 semaphores
were garbage.)

So:

- creating LOTS of semaphores (many thousands) can be expensive, as
   (eventually) freeing them in the GC may be.
- it's hard to precisely define LOTS; it seems to be (as one would
   expect) a factor of what resources are available (I think that
   David and I are both often running on machines with 8GB of physical
   memory; I think that I've seen other types of misbehavior sooner
   on an iBook ...)
- my guess is that very few programs are likely to be affected by
   this and that it'd be counterproductive to agonize over semaphore
   creation. Much. Usually.

>
> --Joe