[Openmcl-devel] Cockpit Error using SAVE-APPLICATION?

[Gary Byers]

On Thu, 6 Nov 2008, j-anthony at comcast.net wrote:

>
> Hi,
>
> Thanks for info and fix for this.  Sadly I have a few more questions :-|
>
> * First, I can pick up the 1.2 for Mac/Darwin just fine, but when I try
> linuxx8664 I get this:
>
> $ svn co http://svn.clozure.com/publicsvn/openmcl/release/1.2/linuxx8664/ccl
> A    ccl/LX86CL64
> A    ccl/lx86cl64
> svn: In directory 'ccl'
> svn: Can't move source to dest
> svn: Can't move 'ccl/.svn/tmp/prop-base/lx86cl64.svn-base' to
> 'ccl/.svn/prop-base/lx86cl64.svn-base': No such file or directory
> $

The two most likely culprits that I can think of are:

1) there's already a directory named "ccl" and "svn co" is getting confused
by its presence.

2) you're on a case-insensitive filesystem (maybe because of running under
VMWare or similar on a host with a case-sensitive filesystem ?) and 
LX86CL64 and lx86cl64 are seen as the same file, also confusing svn.

I suppose that there might be other explanations.


>
> I poked around, but wasn't sure what the "right" thing to do was to fix/get
> around this.  Any ideas?
>
> * I poked around in the source before getting your reply and noticed that
> for a toplevel for class APPLICATION there was a BEFORE method for such
> that included a call to
> (initialize-interactive-streams).  Putting this in a given toplevel
> function got around the error (as you note - the streams weren't
> initialized but hacking it like this "fixed" that issue).  In the process
> of this, I noticed that the default application class
> LISP-DEVELOPMENT-SYSTEM didn't have such a method on it.

Applicable :before methods get called before the primary method, so
this happens when a LISP-DEVELOPMENT-SYSTEM starts up.

It happens fairly late in the initialization process in 1.2  (because
"initializiing interactive streams" involved setting up their character
encoding, which could only be done after parsing command-line arguments
happened, which ddepended on other things ...)  One consequence of this
is that if errors happen very early in the initialization process, something
will try to write an error message to a stream that hasn't been initialized
yet, causing an error, causing .. a trip to the kernel debugger with
little indication of what the real problem was.

"initializing  interactive streams" will be separated from (and happen
earlier than) "setting the character encoding on *TERMINAL-IO*" in the
next release.

> Was this the
> issue?  I notice that 'application is not external in CCL, so subclassing
> an application class off this and using it probably isn't a "right" thing
> to do?

Basically, the :toplevel-function argument to SAVE-APPLICATION gets
run instead of the TOPLEVEL-FUNCTION method of *APPLICATION*; it wasn't
doing the stream initialization and wasn't running the function in
a particularly reasonable environment (^C wasn't detected, FORCE-OUTPUT
wasn't called on *TERMINAL-IO* automatically, etc.)

The changes that I checked in yesterday should make it more reasonable,
but ultimately using an "application" object and methods on it is a
better way of specifying more aspects of a saved application's behavior.
The support for that is incomplete and tends to be a moving target,
and it may still make sense to support the :TOPLEVEL-FUNCTION argument
as a way of generating simple applications.

>
> * It looks like the toplevel function does _not_ control image exit (in the
> sense of a primary thread which if it exits, the whole image shutsdown).  I
> say this because the versions of toplevel which did not encounter an error
> but which exited immediately, didn't seem to affect the running image - it
> just kept running.  In fact, you couldn't even Ctr-c out of it.  Had to
> kill -9 the process explicitly.  Would a (quit) executed from any function
> in the image shut things down?

Calling QUIT should generally shut things down, yes.  The only cases
where it might not involve things like:


(tagbody
   AGAIN
   (unwind-protect
       (do-something)
    (go AGAIN)))

QUIT tries to persuade the initial thread to reset itself and then
kill other threads; if code like the above is running on the initial
thread, it might never get to the point where it's reset itself.

The change that I made yesterday basically does the following, where
USER-TOPLEVEL-FUNCTION is the :TOPLEVEL-FUNCTION argument to SAVE-APPLICATION:

(lambda ()
   (restore-lisp-pointers) ; should have happened already, this should be a nop
   (initialize-interactive-streams)  ; necessary in 1.2
   (process-run-function "toplevel" (lambda ()
                                      (funcall user-toplevel-function)
                                      (quit)))
   (%set-toplevel #'housekeeping-loop) ; force the initial thread to
   (toplevel))                         ;  run CCL::HOUSEKEEPING-LOOP

That basically runs the specified function on a new thread, calls QUIT
if the function exits normally (I suppose that it'd be safer to call
QUIT in an unwind-protect cleanup), and then has the initial thread
do "sleep most of the time, waking up a few times a second to do 
periodic housekeeping tasks, such as forcing output to *TERMINAL-IO*
and checking for ^C and interrupting some thread if it's been
recieved."

That's more reasonable than it was (and your example seems to work),
but there are still some issues:

- if QUIT isn't called after the user function returns, the thread
will die but the initial thread will still sit there waiting for
something interesting to happen.

- a ^C will cause the initial tread to interrupt itself, which
usually isn't very interesting.

- the thread running the user function doesn't "own" the input
side of *TERMINAL-IO*, so things like Y-OR-N-P (or entrering
a break loop) will "request ownership" of that stream (which
is owned by the initial thread, which isn't doing anything
too interesting with it.

>
>
> Sorry for the rambling and extra questions, but I appreciate your efforts
> and any info you may have on this stuff, as I/we are trying to deploy what
> we have as a service.  BTW, CCL works just fine in the Amazon cloud!

What I think people often do (and one reason why this stuff isn't
exercised too well) is something like:

- load your code and save an image without specifying any non-default
startup options

- start the image with an --eval option

shell> ccl64 -I my-image --eval \(start-my-server\)

--eval gets processed after all of the initialization takes place
but before the REPL is entered (and if START-MY-SERVER never returns,
the REPL is never entered.)

This often works fairly well (and has the advantage that things run
in a fairly "normal" environment that's very similar to that in
which the development environment runs.)  It's additionally possible
to run the lisp under "detachtty" (<http://www.cliki.net/detachtty>)
or something similar, so that it runs in the background (and you
can attach to it for debugging.)


>
> Thanks,
>
> /Jon