[Openmcl-devel] Managing multi-threaded access to list

[David L. Rager]

Ah, you are correct.  In my project I don't use the return value all the
time, because I throw threads a lot, requiring some unwinds and the
semaphore notification object.

(loop while *consumers-stay-alive*
  (when (timed-wait-on-semaphore *packet-list-semaphore* 60)
        (consume-packet-no-hang)))

Ah, yes.  Much better.

I forget where the race condition was... I may have been wrong about that,
but there was something going through my mind....  I think the above is fine
though, because consume-packet-no-hang evaluates iff the semaphore is
acquired.

GL,
David


> -----Original Message-----
> From: Joseph Oswald [mailto:josephoswald at gmail.com]
> Sent: Sunday, May 28, 2006 8:09 PM
> To: David L. Rager
> Subject: Re: [Openmcl-devel] Managing multi-threaded access to list
> 
> I'm not sure I follow exactly where the race condition occurs.
> 
> I was planning on having some relatively long (60 seconds or so)
> timeout, and the consumers would check the return value of
> timed-wait-on-semaphore; if they found they had returned because of a
> timeout, they would basically give up on ever receiving a packet, and
> check an error flag on the way out to see if the connection had any
> explanation for the problem.
> 
> The producer, when the connection closed, would set the error flag,
> and depend on the consumers timing out instead of trying to signal
> exactly the right number.
> 
> Or, I could simply signal a "reasonable maximum" number of consumers
> to wake up, realize the error condition, then the rest would simply
> have to be killed manually.
> 
> Thanks again for your help.
> 
> --Joe
> 
> On 5/28/06, David L. Rager <ragerdl at cs.utexas.edu> wrote:
> > Howdy Joseph,
> >
> > Great!
> >
> > The easiest way to know how many consumers are waiting would be to just
> keep
> > track of a count in a lock-guarded and shared variable, say
> > *waiting-consumer-count*.  Then signal the semaphore that many times.
> >
> > Another idea is to create a *maximum-consumer-count*, never create more
> > consumers than that, and signal the semaphore that many times.  I think
> > Darwin PPC limitations are around 300 threads.  My application targets
> 100
> > threads or so.  At some point, I had a target of 200, and that was
> generally
> > stable, but I lowered it for some reason.
> >
> >
> > Using timed-wait-on-semaphore could introduce a race condition
> > (http://en.wikipedia.org/wiki/Race_condition).  If you use
> > timed-wait-on-semaphore, you'll have to use something called a
> > semaphore-notification-object.  Here's an example:
> >
> > (defvar *consumers-stay-alive* t)
> >
> > (let ((SNO (ccl:make-semaphore-notification))
> >   (loop while *consumers-stay-alive*
> >     (progn (ccl:clear-semaphore-notification-status SNO)
> >            (timed-wait-on-semaphore *packet-list-semaphore* SNO)
> >            (when (ccl:semaphore-notification-status SNO)
> >                  (consume-packet-no-hang)))))
> >
> >
> > Timed waiting on a semaphore in OpenMCL has a small downside, in that it
> > shows up as [active] when you type :proc into the prompt.  I think this
> > affects garbage collection somehow, but it's almost certainly irrelevant
> to
> > your project.  On the upside, you get safety properties!!!  I understand
> > this feature to be unique to OpenMCL.
> >
> > I don't remember which version of OpenMCL has the
> > semaphore-notification-objects.  I think 1.0 does, but if not, try the
> > latest CVS bleeding edge sources.  They seem to be reasonably stable on
> > Darwin PPC as of an hour ago.
> >
> > David
> >
> >
> > > -----Original Message-----
> > > From: Joseph Oswald [mailto:josephoswald at gmail.com]
> > > Sent: Sunday, May 28, 2006 3:57 PM
> > > To: David L. Rager
> > > Cc: openmcl-devel at clozure.com
> > > Subject: Re: [Openmcl-devel] Managing multi-threaded access to list
> > >
> > > David--
> > >
> > >   Thanks for the very helpful response. I think that will work for me.
> > >
> > >   A few additional questions for you or the list: there might arise
> > > some condition that indicates that no more packets will be coming in.
> > > Is there a way to know how many consumers were waiting, and to unblock
> > > them (presumably, after they wake up, they will check for this "closed
> > > condition" and go away without a packet)? ccl::semaphore-value?
> > > A workaround might be timed-wait-on-semaphore to allow a periodic
> check.
> > >
> > >   Furthermore, is there any need to destroy semaphores when the
> > > connection goes away? Are they a limited OS resource?
> > >
> > > --Joe
> > >
> > > On 5/25/06, David L. Rager <ragerdl at cs.utexas.edu> wrote:
> > > > Hi Joseph,
> > > >
> > > > You are missing a basic building block: the semaphore.  Also, you
> should
> > > use
> > > > the exported function (ccl:make-lock) instead of the non-exported
> > > > make-read-write-lock.
> > > >
> > > > A semaphore allows for efficient (fast) signaling between threads.
> From
> > > the
> > > > "make-semaphore" documentation:
> >
> >