[Openmcl-devel] Asynchronous callback made from real-time thread

[Gary Byers]

On Fri, 30 May 2003, Letz Stephane wrote:

> Hi,
>
> We are trying to interface a Midi package (MidiShare :
> www.grame.fr/MidiShare) with OpenMCL. Using MidiShare allows a Lisp
> program to access a full Midi API : receive, send  Midi events, tasks
> management...
>
> For real-time handling of incoming Midi events, there is a safe
> solution that consist in doing polling on the incoming event fifo in
> a Lisp thread.
>
> We tried to implement a more real-time way by calling back the Lisp
> code directly from the real-time Midi thread using a lisp callback
> defined with defcallback.
>
> This seems to almost work on 0.13.5 version but crash after some
> time. It crash immediately with the 0.14 alpha version.
>

I have a hunch that the "seems to work on 0.13.5" part means "is
able to run for a while scribbling over some other threads' stacks
before anything actually notices."

Do you remember how it crashes in 0.14 ?  If it dies with a kernel
debugger message saying something like "No TCR for thread ...", that's
actually fairly mundane.  If a thread that lisp didn't create tries to
run a lisp callback, some stacks and other thread-specific data
structures need to be created and the GC needs to be told about the
new thread.  The code to accomplish that isn't too different from the
code that creates a lisp thread in the first place.  (At least I don't
think that it should be: that case isn't handled yet, and all that
happens in that case is a trip to the kernel debugger.  There also
needs to be a little bit of work done on the lisp side in such a case
to establish a lisp process and to establish some special variable
bindings and generally make the thread look more like other lisp
threads ...)

If you're seeing some other type of crash in 0.14, please let me know
what you can about it.  From what you describe, the "No TCR for thread"
error sounds like a likely result.

> Is it possible to implement this kind of asynchronous callback made
> from real-time thread in OpenMCL?

Calling lisp code from any native thread (other than the initial one)
can't possibly work in 0.13.5.

Calling lisp code from native threads is ... what happens all the time
in 0.14.  (If lisp didn't create the thread, there's some lisp-specific
setup that's supposed to happen on the first callback on such a thread;
once that happens, the thread should look like any other.)

It is -very- hard to support reentrancy: if the handler can be invoked
while it's running, that's very hard to deal with at the application
level and even harder to support at the level of the lisp runtime.
(The cases where it can be supported at all require access to the
thread's context - as provided to a signal handler - and require a bit
of care to ensure that atomic operations like memory allocation happen
atomically.)

If the realtime thread is doing something like:

 (loop			; most of this is probably really C code ...
   (wait-for-midi-event)
   (callback-to-lisp-handler))

that shouldn't be a problem (and is very, very similar to what happens
all the time in 0.14)

> Is there any special issue to
> consider : thread synchronization, interaction with the garbage
> collector...
>

Yes there are such issues.  Fortunately, you shouldn't have to deal
with many of them ... (you -do- need to deal with application-specific
locking issues.)

The GC in OpenMCL is fairly quick, but it's not real-time or very
close to it.  It's also not concurrent: it suspends all other threads
while it's running.  I don't know what kind of real-time constraints
might be involved in MIDI programming, but (if and when things are
running reliably) you probably want to minimize consing.

Note that if the MidiShare package allows you to do a blocking wait
on the event fifo (as the real-time thread presumably does), that may
also be a reasonable approach (and one which avoids the "callback on
non-lisp thread" complexity.)

>
> Thanks
>
> Best Regards
>
> Stephane Letz
>

_______________________________________________
Openmcl-devel mailing list
Openmcl-devel at clozure.com
http://clozure.com/cgi-bin/mailman/listinfo/openmcl-devel