[Openmcl-devel] Question about 'process-interrupt run on exhausted process'

[Gary Byers]

First of all, when you run a "standard" OpenMCL image, two lisp threads/
processes are created: the initial thread (created by the OS) creates
a thread to run the listener, then mostly sits idle (it wakes up a few
times a second to force output to interactive streams, check to see
if a ^C interrupt is pending, etc.)

When QUIT is called, the initial process wakes up and tries to get its
peers to exit in an orderly manner (to commit ritual seppuku, but to
try to close files and clean up after themselves).  It does this by
calling PROCESS-KILL on eack other thread, and PROCESS-KILL is
implemented in terms of PROCESS-INTERRUPT.

The initial thread is awoken from its slumber by QUIT: the thread
which calls QUIT uses PROCESS-INTERRUPT to tell the initial thread
to begin the process of shutting down all other threads.

The listener thread (or whichever thread called QUIT in the first place)
has seen no reason to stick around, and has started to exit.

There's obviously a race condition here (well, there are several); the
most common case has to do with whether the listener thread has already
exited by the time that the initial thread wakes up and starts trying
to shut down other threads.  If it has clearly done so, the initial
thread has nothing to do; if it has not, then the initial thread 
issues a PROCESS-KILL to hasten its demise.

A lisp process is "exhausted" if it has no resources (underlying OS
thread, stacks, etc.) which would allow it to run.

The case that you seem to be encountering (and it's amazing that
it doesn't happen more often) could (and probably does) arise from 
a scenario where the listener thread exits ("exhausts itself") between
the time that the initial thread notices that it's still around and
the time that it tries to PROCESS-KILL it.  PROCESS-KILL calls
PROCESS-INTERRUPT, and PROCESS-INTERRUPT notices that there's no
longer any OS thread associated with the target lisp process.

It signals an error (the one that you're seeing) in that case, but
(a) since things are generally happening asyncronously, it's not
clear that this is an error so much as it's a case of bad timing ...
(b) there isn't a whole lot that anyone can do to recover in that
case.

Even if the underlying resources were still there when PROCESS-INTERRUPT
looked for them, there's no guarantee that (for instance) the underlying
OS thread is still valid when PROCESS-INTERRUPT sends it a signal, and
even if it was there's no guarantee that it still exists when the OS
tries to deliver that signal to it.  The most that PROCESS-INTERRUPT
can really do is to return some indication of whether or not it successfully
issued the interrupt request or not, and it doesn't make sense for it
to treat the things that could prevent that request from being delivered
as errors.  (As you've noticed, it makes no sense at all to signal an
error complaining about a dead thread when QUIT is trying to tell the
thread to die ...)

As described in the manual, PROCESS-INTERRUPT's return value isn't very
useful.  A more useful convention would be for it to return NIL if it
was unable to complete the process of issuing an interrupt request and
non-NIL if it was able to completely issue that request.  (The trivial
case of interrupting the current process would return true if it returns
at all.)

This all had something of a familiar ring to it; according to a terse
comment in the bleeding-edge CVS logs, I did make that change shortly
after 1.0 was released, but never backported it to 1.0.

For 1.0, I -think- that it'd be safe to simply remove the check
in PROCESS-INTERRUPT (in ccl/level-1/l1-processes.lisp); in this context,
the check leads to an error that amounts to "I can't kill this dead process!":


(defmethod process-interrupt ((process process) function &rest args)
   "Arrange for the target process to invoke a specified function at
some point in the near future, and then return to what it was doing."
   (let* ((p (require-type process 'process)))
     (if (eq p *current-process*)
       (apply function args)
        (let* ((thread (process-thread p)))
          #+want-to-hear-about-inability-to-interrupt-dead-processes
          (when (thread-exhausted-p thread)
            (error "process-interrupt run on exhausted ~s" p))
          (progn
 	   (thread-interrupt
 	    thread
 	    process
             #'apply
 	    function args))))))







On Sat, 10 Feb 2007, Denis Bueno wrote:

> Dear list:
>
> I'm having a problem when executing openmcl as a child process of some
> OCaml code I'm writing. I am writing a compiler for a language which
> embeds Common Lisp (let (...) ...) fragments that return integers.
>
> I know that each let-form is well-formed and results in an integer.
> For each let-form in the source I'm compiling, I create one openmcl
> process, evaluate the let form, read back the result, and kill off the
> process. This is all done from a single thread, so no two openmcl
> processes should be alive at the same time.
>
> However, *sometimes* (maybe 1 time every 100 or so openmcl processes)
> I get the following error
>
> ,----
> | > Error: process-interrupt run on exhausted #<PROCESS listener(1)
> [Exhausted] #x1046DB06C>
> | > While executing: #<CCL::STANDARD-KERNEL-METHOD PROCESS-INTERRUPT
> (PROCESS T)>
> |
> | ;; not sure the ordering of these two outputs of OpenMCL
> |
> | 7
> | > Type :POP to abort.
> | Type :? for other options.
> `----
>
> (The note inside the fragments above simply means that since the top
> half of the errors occurs in the terminal running the unit tests, and
> the bottom half occurs in the log of the file being tested, and both
> come from the openmcl process, I'm not sure which text comes first. I
> assume someone else will, or it won't matter.)
>
> The output above is generated with a certain shell command [1]. Does
> anyone know what could cause the above? Is there a way to avoid it?
>
> Thanks in advance.
>
> -Denis
>
> [1]
> openmcl --quiet --no-init --batch --eval '(handler-case    (progn
> (print (let ((c1 2)
> (c2 2)
> (d1 -2)
> (d2 -3)
> )
> (ceiling (sqrt (+ (expt (- d1 c1) 2) (expt (- d2 c2) 2))))))
> (finish-output) (quit))    (process-reset () (finish-output) (quit))
> (t (cond) (progn (princ cond) (finish-output) (quit))) )'
> _______________________________________________
> Openmcl-devel mailing list
> Openmcl-devel at clozure.com
> http://clozure.com/mailman/listinfo/openmcl-devel
>
>