[Openmcl-devel] debugging debugging

[Gary Byers]

On Fri, 8 May 2009, Arthur W Cater wrote:

> As I mentioned in March, I have a similar problem with errors occurring on other threads,
> in my case often threads other than the Event thread.
> I'm told to type (:Y 8)   eg   but neither the Listener nor the AltConsole does anything
> when I do so. A simple example of this is with
>
> (process-run-function "Breaker" #'(lambda nil (break "Talk to me")))
>
> The Event thread still runs, but there is no way afaik of getting any
> debugging info, not even a simple backtrace.
>
> Arthur
>

Yuck.  Let me see if I can explain what's going on here (though I agree that
what's going on and what should/could be going on are different things.)

When the lisp starts up, it initializes (global) values of the standard input
and output stream variables.  When run in a TTY environment or under Emacs,
these streams are (usefully) connected to the process-wide standard input
and output devices (pipes or sockets or terminal-like devices).  When a GUI
application is launched on OSX, the devices to which these streams are connected
aren't appropriate for interactive I/O; when the CCL IDE starts up, it tries
to reconnect these streams to devices that talk to the AltConsole application
(which at least provides a primitive way of doing interactive I/O.)

When the (non-GUI) CCL starts up, a listener thread is created that uses
these standard streams (and can therefore talk to a Terminal window, Emacs, etc.)
The listener thread is marked as being the "owner" of the global standard input
stream (since it's expected to make heavy use of that stream, to run the REPL,
etc.)  If some background thread needs to do input from that stream, it announces
that fact and advises us to type (:y <n>); in the listener, we have to ensure
that we're talking to the REPL (either at top level or in a break loop) and type
that command in order to (temporarily) transfer the exclusive right to use that
global input stream to the background thread.

If the listener thread isn't in the REPL, we generally have to wait until it
is or interrupt it to get it to that point before typing (:Y <n>) will have
the desired effect.

When the IDE starts up, the initial thread is marked as the owner of the global
input stream (connected to AltConsole).  When listener windows are created in
the IDE, the listener threads that are created at the same time arrange to use
interactive streams connected to that listener window (and its buffer.)  If
a background thread is created, then by default it will try to use the global
(AltConsole-based) interactive streams.  (See below.)

If that background thread needs to use the global input stream (owned
by the initial/event thread), it'll print a message to that effect on
its standard error or output stream (I forget which, but they'd both
be associated with AltConsole) and wait for the input stream's owner
to transfer ownership (via a :y command in a REPL).  Of course,
the input stream's owner is busy processing events isn't really running
a REPL, so (as you noted) nothing really happens.

If we were to keep this paradigm (background threads try to do emergency
I/O using a shared input stream), then we'd presumably want to have some
other means of doing the equivalent of :Y (a menu command or something)
that wouldn're require the event thread to be at a REPL prompt.  If you
want to see what's going on here or need to handle this situation in an
emergency in the short term, you can do:

- in an IDE listener, do:

? (process-run-function "background" #'break)

This should cause output to the AltConsole window, announcing that the background
thread wants to enter a break loop and needs someone to type a :Y command to let
it use the global input stream.  (Of course, it's not the act of typing that
command that transfers ownership, it's the act of a REPL running in the owning
thread reading and processing that command that does the transfer.)  The owning
thread, of course, is busy processing events, so let's change that.

- in the IDE listener, do:

? (process-interrupt ccl::*initial-process* #'break)

There are several ways to refer to the initial process; sadly, I don't know
of a way that involves exported names.

Event processing should stop, and the initial process should have entered a
break loop in the AltConsole window.  In that break loop typing the appropriate
:Y command will do the "ownership transfer" and let the background thread use
the input stream for its break loop; exiting the break loop will restore control
to the event thread (which will be in the break loop we forced it into from the
IDE), and exiting from that break loop will allow the event thread to get back
to processing events.

That's all pretty horrible; it could be made slightly less horrible if
there were some GUI object (a menu item or button or something
somewhere) that avoided the need to enter a break loop to do the
ownership transfer of the input stream, but ... well, there's no
reason for a background thread to have to use the global
AltConsole-based stream for incidental or other I/O: it could create a
listener window ("on demand") if it needed to do I/O, or try to share
an existing listener window, or (maybe) create its own AltConsole
window, or any number or other things.  (It's very hard to get the
event thread to do event-driven I/O to and from a GUI window, but shouldn't
be any harder for a random thread to do that than it is for a listener.)