[Openmcl-devel] Improving bordeaux-threads on CCL

[Gary Byers]

On Sun, 8 Sep 2013, James M. Lawrence wrote:

> Is ccl:wait-on-semaphore interrupt-safe?

To the extent that I understand your use of the term: yes.
If interrupts are enabled, WAIT-ON-SEMAPHORE can be interrupted; if the
interrupt returns normaly, WAIT-ON-SEMAPHORE will resume an indefinite
wait and a TIMED-WAIT-ON-SEMPAHORE will resume waiting until the original
absolute time or return NIL if that time is in the past.

Code that runs in response to the interrupt can signal or start a new
wait on the semaphore.  (It may or may not be a good idea for that code
to do so, but nothing is going to prevent that code from doing either of
those things.)

All of this needs to be qualified somewhat:  PROCESS-INTERRUPT can be
used to do arbitrary things; that includes stupid things and deliberately
malicious things.  (Essentially the same could be said of a lisp REPL;
you have enough rope to hang yourself, and that's arguably preferable
to not having enough rope to do something that you need to do.)  This
is no more or less true of interrupting WAIT-ON-SEMAPHORE than of anything
else.

> If it is then my next
> question relates to making bordeaux-threads safer. It may be that
> condition variables cannot be perfectly implemented with semaphores,
> but the current situation may at least be improvable. The
> bordeaux-threads code is basically
>
> (defun make-condition-variable ()
>  (ccl:make-semaphore))
>
> (defun condition-notify (condition-variable)
>  (ccl:signal-semaphore condition-variable))
>
> (defun condition-wait (condition-variable lock)
>  (ccl:release-lock lock)
>  (unwind-protect
>       (ccl:wait-on-semaphore condition-variable)
>    (ccl:grab-lock lock t)))
>
> A safe version of condition-wait might be
>
> (defun condition-wait (condition-variable lock)
>  (ccl:without-interrupts
>    (ccl:release-lock lock)
>    (unwind-protect
>         (ccl:with-interrupts-enabled
>           (ccl:wait-on-semaphore condition-variable))
>      (ccl:grab-lock lock))))
>
> However the grab-lock has the potential of causing an unkillable
> deadlock, and there may be other issues.

Since UNWIND-PROTECT cleanup forms are effectively run with interrupts
disabled in CCL by default, the same is true of the call to GRAB-LOCK in
the original code and it's essentially true of things like pthread_cond_wait()
AFAICT.

Anyway: yes, the original version of the code could get interrupted
after releasing the lock and could abandon the attempt to wait; the
lock would have to be reaquired before it could be released on the
next call to CONDITION-WAIT (using that lock).  I don't know that
anything more severe than that can happen with the less-safe version,
but it's surely better to guard against that to not do so.


>
> Interrupting threads is generally bad news because Lisp code is not
> generally expected to be interrupt-safe.

Huh.  I'd generally say that that's more true of C code than Lisp code,
but I'd also find it hard to make sweeping generalizations like that.

Perhaps different people have different general expectations.


> But in the context of limited
> guarantees on particular data structures, an interrupt-safe
> condition-wait would be useful. It could be that such data structures
> on CCL must be written without the condition-variable emulation.

Since CCL doesn't provide condition variables, code would have to continue
to muddle along using locks and semaphores to provide serialization and
synchronizaton.

Aside from a bizarre and somewhat heavyweight interface to similar
functionality, true condition variables do offer one thing that
semaphores generally don't: pthread_cond_broadcast() can be used to
wake up all threads that're waiting for a semaphore-like object.  That
can certainly be useful, but omplementing it likely needs to be done
with great care (and possibly with the cooperation of the OS.)