[Openmcl-devel] thread overview

[Gary Byers]

On Sun, 22 Aug 2004, Douglas Philips wrote:

> All this talking of threading reminded me... where do I find out how
> signals (say, oh, Control-C, depending on your terminal bindings) are
> supposed to be handled? I've noticed that in tight computation loops
> that they don't seem to...


? (defun foo ()
    (loop))  ; a very tight computation loop
FOO
? (disassemble #'foo)
  ;[a few instructions elided]
L12
  (B L12)
  ;[a few more instructions elided]

The empty (LOOP) compiles to a branch instruction that will be executed
forever (or until it's interrupted).

? (foo)

A ^C at this point should force a break loop under most circumstances.
(It certainly should if this is all happening in a terminal window;
under ILISP, it may be necessary to use ^C^C.  I'm not sure about SLIME,
and (of course) I'm assuming that the TTY device maps SIGINT to ^C.)

How this works:

The lisp kernel installs a handler for SIGINT.  That handler is called
in an arbitrary thread (it seems in practice to be the initial thread,
but I don't believe that the OS guarantees this.)  The handler just
sets a flag and returns.

One of the housekeeping activities that some lisp thread is supposed
to perform a few times per second is to check (and clear) that flag.
If the flag was set, the housekeeping thread selects some (other)
lisp thread and uses PROCESS-INTERRUPT to force that thread to enter
a break loop.

[Do you want to know how PROCESS-INTERRUPT works ?]

The "victim" lisp thread (the one that gets PROCESS-INTERRUPTed) is:

  a) the value of CCL::*INTERACTIVE-ABORT-PROCESS*, if that's non-NIL
     (this is supposed to be for the benefit of things like SLIME,
     which typically don't interact with the initial listener.)
     I don't know how SLIME currently handles the issue of
     interruption.
  b) the thread which currently "owns" the default terminal input
     stream (the one associated with file descriptor 0), otherwise.
     (Usually, this is the initial listener.)

If you do something like:

? (defun foo2 ()
    (without-interrupts
      (loop)))

and try to interrupt a call to (FOO2) with SIGINT, exactly the same
set of things will happen, but the target process won't acknowledge
the PROCESS-INTERRUPT until the outermost WITHOUT-INTERRUPTS exits.
Of course, that'll never happen.

For the sake of argument, foreign code is effectively WITHOUT-INTERRUPTS;
most blocking system calls will return prematurely if they receive a
signal during their execution (some wrappers around Mach system calls
restart the system call rather than returning from the wrapper.)


> (this is both a "how does it work now?" and "how should it work?"
> question).

I haven't experienced or heard reports of cases where computations that
should be interruptible via ^C aren't interruptible; in most respects,
I think that the way that it works now is pretty close to the way that
it should work.

Clearly, the thread that -should- receive an interrupt when you press
^C is the one that you're thinking of.  Until the brain-scanner works
more reliably, the decision has to be made more arbitrarily.

Without going into too many details, PROCESS-INTERRUPT works by sending
signals between threads (via #_pthread_signal)  There are some arguments
in favor of doing it in other ways under Darwin (mostly to make
asynchronous interruption of things that make heavy use of Mach
practical.)

Treating foreign code as if it were within a WITHOUT-INTERRUPTS is
probably overly conservative: -some- interesting foreign code isn't
reentrant, but if we knew for sure that that code was reentrant I don't
-think- that there are GC or other issues. It might be possible to
advertise that a particular foreign function call is reentrant - something
like:

((#_harmless :interruptible t) a b c) ; yes, that's pretty ugly ...

and overcome this limitation when practical.

The housekeeping thread - which is responsible for noticing the pending
SIGINT and PROCESS-INTERRUPTing the target thread - does something like:

 (loop
   (sleep .33)
   (handle-control-C-and-other-things))

There can be some latency there, and there can be some latency
involved in PROCESS-INTERRUPT.  I don't think that that latency's
usually high.  I think that the kernel-level SIGINT handler could
probably be moved partly or entirely out of the kernel; reducing
latency here seems to be the most compelling reason for doing so, and
there are other sources of latency involved in signal delivery that
don't have much to do with lisp.

> Thanks,
> 	<D\'gou