[Openmcl-devel] Asynchronous callback made from real-time thread

[Letz Stephane]

>On Fri, 30 May 2003, Letz Stephane wrote:
>
>>  Hi,
>>
>>  We are trying to interface a Midi package (MidiShare :
>>  www.grame.fr/MidiShare) with OpenMCL. Using MidiShare allows a Lisp
>>  program to access a full Midi API : receive, send  Midi events, tasks
>>  management...
>>
>>  For real-time handling of incoming Midi events, there is a safe
>>  solution that consist in doing polling on the incoming event fifo in
>>  a Lisp thread.
>>
>>  We tried to implement a more real-time way by calling back the Lisp
>>  code directly from the real-time Midi thread using a lisp callback
>>  defined with defcallback.
>>
>>  This seems to almost work on 0.13.5 version but crash after some
>>  time. It crash immediately with the 0.14 alpha version.
>>
>
>I have a hunch that the "seems to work on 0.13.5" part means "is
>able to run for a while scribbling over some other threads' stacks
>before anything actually notices."

Yes exactly....
,

>
>Do you remember how it crashes in 0.14 ?  If it dies with a kernel
>debugger message saying something like "No TCR for thread ...", that's
>actually fairly mundane.

It gives this kind of messages:

Thread 6 Crashed:
  #0   0x00122de4 in tsd_get (thread_manager.c:361)
  #1   0x00123850 in get_tcr (thread_manager.c:681)
  #2   0x00123ac4 in suspend_other_threads (thread_manager.c:745)
  #3   0x001215e0 in Bug (lisp-exceptions.c:2442)
  #4   0x00123888 in get_tcr (thread_manager.c:688)
  #5   0x00123ac4 in suspend_other_threads (thread_manager.c:745)
  #


>If a thread that lisp didn't create tries to
>run a lisp callback, some stacks and other thread-specific data
>structures need to be created and the GC needs to be told about the
>new thread.  The code to accomplish that isn't too different from the
>code that creates a lisp thread in the first place.  (At least I don't
>think that it should be: that case isn't handled yet, and all that
>happens in that case is a trip to the kernel debugger.  There also
>needs to be a little bit of work done on the lisp side in such a case
>to establish a lisp process and to establish some special variable
>bindings and generally make the thread look more like other lisp
>threads ...)
>
>If you're seeing some other type of crash in 0.14, please let me know
>what you can about it.  From what you describe, the "No TCR for thread"
>error sounds like a likely result.
>
>>  Is it possible to implement this kind of asynchronous callback made
>>  from real-time thread in OpenMCL?
>
>Calling lisp code from any native thread (other than the initial one)
>can't possibly work in 0.13.5.
>
>Calling lisp code from native threads is ... what happens all the time
>in 0.14.  (If lisp didn't create the thread, there's some lisp-specific
>setup that's supposed to happen on the first callback on such a thread;
>once that happens, the thread should look like any other.)

Could you explain this a little more?

>
>It is -very- hard to support reentrancy: if the handler can be invoked
>while it's running, that's very hard to deal with at the application
>level and even harder to support at the level of the lisp runtime.
>(The cases where it can be supported at all require access to the
>thread's context - as provided to a signal handler - and require a bit
>of care to ensure that atomic operations like memory allocation happen
>atomically.)

I don't think that we have to support reentrancy on the Lisp side. 
The real-time C Midi handler already deal with reentrancy.


>
>If the realtime thread is doing something like:
>
>  (loop			; most of this is probably really C code ...
>    (wait-for-midi-event)
>    (callback-to-lisp-handler))

There is no blocking (wait-for-midi-event) function of the C side. 
But it can probably be implemented using a condition variable.

>
>that shouldn't be a problem (and is very, very similar to what happens
>all the time in 0.14)
>
>>  Is there any special issue to
>>  consider : thread synchronization, interaction with the garbage
>>  collector...
>  >
>
>Yes there are such issues.  Fortunately, you shouldn't have to deal
>with many of them ... (you -do- need to deal with application-specific
>locking issues.)
>
>The GC in OpenMCL is fairly quick, but it's not real-time or very
>close to it.  It's also not concurrent: it suspends all other threads
>while it's running.  I don't know what kind of real-time constraints
>might be involved in MIDI programming, but (if and when things are
>running reliably) you probably want to minimize consing.

Is there a way to disable GC during the realtime callback?

>
>Note that if the MidiShare package allows you to do a blocking wait
>on the event fifo (as the real-time thread presumably does), that may
>also be a reasonable approach (and one which avoids the "callback on
>non-lisp thread" complexity.)
>

Having a blocking wait on the event fifo is probably possible, but 
i'm interested also to know more about the "complex" solution (:

Thanks

Stephane Letz

_______________________________________________
Openmcl-devel mailing list
Openmcl-devel at clozure.com
http://clozure.com/cgi-bin/mailman/listinfo/openmcl-devel