[Openmcl-devel] Cockpit Error using SAVE-APPLICATION?

[Gary Byers]

On Fri, 7 Nov 2008, j-anthony at comcast.net wrote:

>> The single thread will own the right to read from the input side of
>> *TERMINAL-IO*.
>
> What are the practical ramifications of this?

Basically, unless some other mechanism's in place (a window system,
network streams), all threads share the same value of *TERMINAL-IO*
(and other standard streams which are aliased to *TERMINAL-IO*.)  It
two threads try to write to the output side if *TERMINAL-IO* at
roughly the same time and don't take other steps to synchronize with
each other, the output may look messy (chunks of output from different
threads may be mixed together), but nothing too catastrophic happens
(actual access to/modification of the internal state of a shared
stream involves locking, so that state should be updated consistently.)

If two threads try to read from *TERMINAL-IO* at roughly the same
time, it's also thread-safe, but it's unpredictable which thread will
receive any pending input.  (E.g., if two threads call READ-CHAR on
the same stream at "about the same time" and two characters of data
are available, we can't predict which thread will receive which (if
any) of those available characters; we need some higher-level way
of deciding which thread has the right to read from the shared stream
and some mechanism for transferring that right.  (This is similar
to the situation that occurs when multiple Unix processes are running
in the background in a Unix shell; most shells provide some form
of job control, so that selected processes/shell jobs can be put
in the foreground/background; the shell generally gets a signal
when a child process that's in the background tries to read from
the shared tty.)

The way that this works in CCL is that some thread "owns" (or "is the
primary owner of") the shared terminal input stream; if some other
thread needs to read from it, it announces that fact and waits for the
(primary) owning thread to transfer ownership to it; when the "background"
thread is done with the stream, it causes ownership to revert to the
primary owner.

When the initial thread (the one created by the OS when the OS-level
process starts up), it's the primary owner of the shared input stream.
When the listener thread (in a LISP-DEVELOPMENT-SYSTEM) starts up and
throughout its entire lifetime, it becomes the owner; when the
listener thread exits, it transfers ownership back to the initial
thread.  (So except for the brief periods before the listener thread
is created and after it exits, the listener thread is the primary
owner of the shared terminal input stream.)

Break loops, Y-OR-N-P, YES-OR-NO-P, and possibly a few other things
use WITH-TERMINAL-INPUT to request "temporary ownership" of terminal
input; arbitrary calls to READ-CHAR/READ/READ-LINE/etc. don't do so.

So ... my comment meant that since you're running in a single thread
(the initial thread), you don't have to worry about this.  If you
were to run in a second thread (perhaps to allow the initial thread
to do the "housekeeping loop"), you'd have to jump through the
same hoops that the listener initialization code jumpe through
in order for your application's thread to be the "primary owner"
of the shared input stream.


>
>
>> You can get that (and the automatic flushing of output to *TERMINAL-IO*)
>> to happen by starting a thread to run CCL::HOUSEKEEPING-LOOP.  That
>> wont add too much overhead (the thread will spend 99+% of its time
>> sleeping.)
>>
>> (defmethod ccl::toplevel-function ((a aprigo-application) x)
> ...
>>  (PROCESS-RUN-FUNCTION "background" #'CCL::HOUSEKEEPING-LOOP)
>
> Is this basically equivalent to what
> (defmethod toplevel-function ((a lisp-development-system) init-file)
> does with the last two things it does:
>
> (%set-toplevel #'housekeeping-loop)
> (toplevel)
>
> Or maybe it is just the %set-toplevel call that equates to your
> "background" process example above?  (Actually, (toplevel) is a bit of a
> mystery to me - I tried tracking it down a bit but seems to involve some
> sort of "throw :toplevel ... catch :toplevel" idiom/technique.)
>

The two approaches - letting your code run on the initial thread
and having another thread do the housekeeping loop (which does
periodic FORCE-OUTPUT to the output side of *TERMINAL-IO* and
^C handling) and running your code on a second thread and having the
initial thread do the housekeeping loop are mostly equivalent.  If
you stay on the initial thread, you don't have to worry about
the terminal-input ownership issue.

TOPLEVEL and %SET-TOPLEVEL exist mostly because the initial thread
(the one created by the OS) is different from things created by
PROCESS-RUN-FUNCTION and friends and it's harder to manipulate it
with PROCESS-PRESET/PROCESS-RESET/etc.  When a thread starts
up (or when the initial thread is finished loading the image),
it calls a function in the lisp kernel that does something like:

(loop
   (catch 'some-tag
     (let* ((fn (current-thread's-initial-function)))
        (if fn
          (funcall-fn)
          (return)))))

CCL::%SET-TOPLEVEL sets what I referred to as the current thread's
initial function to a 0-argument function or NIL.  CCL::TOPLEVEL
throws to that tag and either funcalls the initial function or
exits the loop (the thread basically dies soon after it exits
that loop.)


>
>> Ideally, it'd be possible to easily define -some- methods on
>> a subclass of CCL::LISP-DEVELOPMENT-SYSTEM so that everything
>> but the invocation of the REPL happened via the superclass's
>> methods and you just ran some other code instead of the
>
> Wouldn't this basically be what the standard toplevel does in creating the
> listener process with the call:
>
> (startup-ccl (and *load-lisp-init-file* init-file))
>
> And then you would not follow this with the call it also does:
> (listener-function).
>
> Sounds a bit hacky, but would this pretty much do the whole thing except
> the REPL?
>

Yes, although it's pretty ugly to have to clone the TOPLEVEL-FUNCTION
method for LISP-DEVELOPMENT-SYSTEM and just change some or all of
the function passed as the value of :INITIAL-FUNCTION.

If there was some other method there or if things were modularized
a little better:

(defmethod think-of-a-good-name-for-it ((application lisp-development-system))
   (startup-ccl ...)
   (listener-function)
   nil)

(defmethod toplevel-function ((application lisp-development-system))
   ...
   :initial-function (think-of-a-good-name-for-it *application*)
   ...)

you'd just need to subclass LISP-DEVELOPMENT-SYSTEM and define
that method that I can't think of a good name for on the subclass ...

>
> /Jon
>
>
> --------------------------------------------------------------------
> mail2web.com - Microsoft® Exchange solutions from a leading provider -
> http://link.mail2web.com/Business/Exchange
>
>
>