[Openmcl-devel] *initial-process* and *current-process*

[Gary Byers]

SAVE-APPLICATION basically just writes the contents of the lisp heap
to a file where it can (conceptually) be read back in.  It's more
complicated in some of its details, but that's not an unreasonable
model of what it does.

You might (in some way) be able to define what it would mean to
save dynamic thread state (stacks, etc.) so that it could be restored
when the image started up - so that in some way threads were preserved
and could sort of resume what they were doing when SAVE-APPLICATION
was called.  I've never been able to figure out just what it'd mean
for a thread to (dynamically) survive across sessions.  (If anyone
could figure out what that means and how to implement it, I can sort
of imagine that it might be useful.)

CCL's SAVE-APPLICATION avoids the issue (of saving thread state) completely:
it basically causes the initial thread to reset itself and try to shut down
all other threads in a (hopefully) orderly manner (running UNWIND-PROTECT
cleanup forms and undoing dynamic bindings).  The code that runs at this
point is doing essentially what's done when CCL:QUIT is called; the only
real difference between SAVE-APPLICATION and QUIT is that SAVE-APPLICATION
arranges to write the contents of the lisp heap to a file as the last
thing it does before exiting.  That last act occurs in an environment
where there are no threads but the initial thread and no lisp code is
running.

A dynamic binding (a binding of a special variable via
LET/LAMBDA/PROGV/etc) is inherently thread-local.  SAVE-APPLICATION
basically runs in a single-threaded environment; starting up with that
saved image basically starts up in a roughly equivalent single-threaded
environment; the "toplevel function" specified/defaulted in the call
to SAVE-APPLICATION might create new threads (that might be functionally
similar to the threads that existed when the image was saved.)  Those
new threads won't have the "same" dynamic bindings in effect that the
similar threads had before the image was saved.

In other words:


;; We might say that there's a "static" binding of *x* to 1.
(defvar *x* 1)


;; In the listener thread
(let* ((*x* 17)) ; a dynamic binding of *x* to 17
   (save-application "x-is-17.image"))

;; By the time SAVE-APPLICATION actually gets around to writing the image.
;; the listener thread will be dead; as it dies, it disestablishes its 
;; dynamic bindings.

$ ccl-or-ccl64-its-so-confusing -I x-is-17.image
Welcome to whatever version this is!
? *x*
1

It's true that some lisp PROCESS objects persist across
SAVE-APPLICATION; I think that this is true of the value of
*INITIAL-PROCESS* and it's also true of the value of some variable
that refers to the initial listener process.  That allows us to say
things like "this thread-local hash table is owned by the initial
listener" and have that ownership persist, but nothing "under the
hood" of that PROCESS object really persists: there's a new OS-level
thread involved, with stacks whose size/location/content all likely
differ and whose dynamic bindings and other aspects are only
coincidentally similar.

Dynamic bindings aren't saved in the image; the binding (the association
between a variable and a value) is thread-local, and the dynamic state
of threads (a) doesn't exist when the image is written and (b) therefore
isn't written to the image.

This has never worked differently in CCL.  (I would know if it had.  Honest.)

If the example above had been written as:

(setq *x* 17) ; modify the static binding

(save-application ...)

then that modification would persist.

On Thu, 23 Jun 2011, Gabriel Dos Reis wrote:

> On Thu, Jun 23, 2011 at 12:42 AM, Gary Byers <gb at clozure.com> wrote:
>> On Wed, 22 Jun 2011, Gabriel Dos Reis wrote:
>>
>>> Hi,
>>>
>>> I have an application where when I use ccl:save-application,
>>> CCL ?apparently saves the "old image" (not what I want), that is the image
>>> as it was before I LOAD some fasls instead of the "current image" that
>>> contains
>>> the loaded fasls. ?I started seeing that behavior only with "recent"
>>> versions of CCL --
>>
>> I have no idea how SAVE-APPLICATION would do this. ?How would it know
>> what was part of the "old image" and what wasn't ?
>
> It could be looking at argv[0] in a special way :-)  I dunno, which is
> why I was asking.
> (Some Lisps such as CLisp and GCL interpret argv[0] specially.)
>
>>
>> There are some scenarios where SAVE-APPLICATION could start to write
>> an image file and encounter some sort of catastrophic error (out of
>> disk space, memory corruption, a bug) before finishing. ?The last
>> thing written to an image file is signature information that the image
>> loader uses to validate the file; an incomplete image file
>> can't be loaded.
>
> Thanks. That scenario does not seem to be what is happening in my case.
> I tried (most part of the night) to reduce the situation to a simple
> reproducible case,
> but failed so far.
>
>> You might want to check pathnames and file write dates carefully;
>> I don't think it's likely ?that what you think is happening is happening,
>> and it seems much more likely that there's a more mundane explanation
>> for what you're seeing.
>
> I am saving the application in a directory different from the one containing
> the original image.  I print the state the objects I interested in right before
> SAVE-APPLICATION (they have the right updated values.) When I print
> the same objects right after resuming the saved image, they have the old
> values before modification and saving.
>
>>> that is anything but recent CCL from trunk works fine.
>>
>> I think that you'll find that recent versions of the trunk also work
>> fine and that you're misunderstanding what you're seeing.
>
> that is possible; I'm still investigating.
> Everything works fine with the Windows image available from SVN.
> However, it fails on linux 64-bit and I did not introduce anything
> platform specific when saving.
>
>>> That led me to wonder the difference between *INITIAL-PROCESS* and
>>> *CURRENT-PROCESS*. ?Thanks,
>>
>> As it's used in the names of these variables, the word "process"
>> essentially means "thread" or "lightweight process". ?The initial
>> process (the lisp process/thread that's the value of
>> *INITIAL-PROCESS*) was created by the OS when the application started,
>> and (as David Rager said) each thread has its own dynamic binding of
>> *CURRENT-PROCESS* that refers to itself.
>
> Thanks to both of you.
> The objects I am interested in are bound to dynamic variables.  Is it
> at all possible
> that a separate thread  is performing SAVE-APPLICATION and therefore
> is seeing different different values of the dynamic variables?
>
> Thanks,
>
> -- Gaby
>
>