[Openmcl-devel] Managing multi-threaded access to list

[David L. Rager]

Howdy Joseph,

Great!

The easiest way to know how many consumers are waiting would be to just keep
track of a count in a lock-guarded and shared variable, say
*waiting-consumer-count*.  Then signal the semaphore that many times.

Another idea is to create a *maximum-consumer-count*, never create more
consumers than that, and signal the semaphore that many times.  I think
Darwin PPC limitations are around 300 threads.  My application targets 100
threads or so.  At some point, I had a target of 200, and that was generally
stable, but I lowered it for some reason.


Using timed-wait-on-semaphore could introduce a race condition
(http://en.wikipedia.org/wiki/Race_condition).  If you use
timed-wait-on-semaphore, you'll have to use something called a
semaphore-notification-object.  Here's an example:

(defvar *consumers-stay-alive* t)

(let ((SNO (ccl:make-semaphore-notification))
  (loop while *consumers-stay-alive*
    (progn (ccl:clear-semaphore-notification-status SNO)
           (timed-wait-on-semaphore *packet-list-semaphore* SNO)
           (when (ccl:semaphore-notification-status SNO) 
                 (consume-packet-no-hang)))))


Timed waiting on a semaphore in OpenMCL has a small downside, in that it
shows up as [active] when you type :proc into the prompt.  I think this
affects garbage collection somehow, but it's almost certainly irrelevant to
your project.  On the upside, you get safety properties!!!  I understand
this feature to be unique to OpenMCL.

I don't remember which version of OpenMCL has the
semaphore-notification-objects.  I think 1.0 does, but if not, try the
latest CVS bleeding edge sources.  They seem to be reasonably stable on
Darwin PPC as of an hour ago.

David


> -----Original Message-----
> From: Joseph Oswald [mailto:josephoswald at gmail.com]
> Sent: Sunday, May 28, 2006 3:57 PM
> To: David L. Rager
> Cc: openmcl-devel at clozure.com
> Subject: Re: [Openmcl-devel] Managing multi-threaded access to list
> 
> David--
> 
>   Thanks for the very helpful response. I think that will work for me.
> 
>   A few additional questions for you or the list: there might arise
> some condition that indicates that no more packets will be coming in.
> Is there a way to know how many consumers were waiting, and to unblock
> them (presumably, after they wake up, they will check for this "closed
> condition" and go away without a packet)? ccl::semaphore-value?
> A workaround might be timed-wait-on-semaphore to allow a periodic check.
> 
>   Furthermore, is there any need to destroy semaphores when the
> connection goes away? Are they a limited OS resource?
> 
> --Joe
> 
> On 5/25/06, David L. Rager <ragerdl at cs.utexas.edu> wrote:
> > Hi Joseph,
> >
> > You are missing a basic building block: the semaphore.  Also, you should
> use
> > the exported function (ccl:make-lock) instead of the non-exported
> > make-read-write-lock.
> >
> > A semaphore allows for efficient (fast) signaling between threads.  From
> the
> > "make-semaphore" documentation: