[Openmcl-devel] async signals

[Gábor Melis]

Better late than never, sbcl-devel is CC'ed to unify the threads.

On Saturday 17 September 2005 00:45, Gary Byers wrote:
> On Fri, 16 Sep 2005, [iso-8859-1] Gábor Melis wrote:
> > On Thursday 01 September 2005 10:47, Gary Byers wrote:
> >> Every time that I've tried to convince myself that unwinding
> >> should ignore or defer asynchronous interrupts - and have
> >> convinced myself that I understood just when unwinding started,
> >> when it ended, and when it was prematurely exited - I convince
> >> myself that that's undesirable and less tractable than it'd first
> >> seemed.  I'm currently on the latter end of that pendulum, but it
> >> was swinging the other way a few weeks ago.
> >
> > Heh, that's familiar.
> >
> >>> I've read that openmcl checks for pending interrupts at
> >>> safe-points.
> >>
> >> Where, out of curiousity, did you read this ?  (It's not true, but
> >> it used to be.  Lisp code can interrupted basically at any time
> >> (unless WITHOUT-INTERRUPTS is in effect.)  The "internals"
> >> documentation on the website is several years old and is maybe 50%
> >> accurate; I think that there's a disclaimer to that effect
> >> somewher
> >
> > Sorry, I read everything under the sun (or google), but I can't
> > recall where it was.
> >
> > I have an implementation for sbcl that's mostly working, with the
> > design decisions detailed in this mail:
> >
> > http://www.caddr.com/macho/archives/sbcl-devel/2005-9/6018.html
> >
> > Come to think of it, I should have probably CCed openmcl-devel in
> > that mail. It is by no means final, comments are welcome.
> >
> > Cheers, Gábor
>
> When I'm on the other end of the pendulum (perhaps that last happened
> this morning), I convince myself that there are two mostly disjoint
> but equally important concepts (that presumably would lead to an
> elegant solution, the details of which are left as an exercise.)
>
> 1) When people first started to think about this (in the context of
> hardware interrupts), they reached the conclusion that it was
> important to assign priorities to interrupts.  To answer the question
> "which is more important: finishing closing the file in an
> UNWIND-PROTECT cleanup form, or acknowledging the user who's been
> pressing ^C for the last minute and is reaching for the power switch
> ?", you may need to assign arbitrary weights to those events and
> use those weights to implement a policy.
>
> I think that commercial MCL had some sort of notion of multiple
> interrupt levels at one point in its history; I don't remember if
> that was ever implemented in a released version.  My recollection is
> that it was abandoned because it was too complicated; I don't
> remember whether that meant "too complicated to implement correctly",
> "too complicated to use", or both.
>
> In hardware interrupt priority schemes, it's often (usually ? always
> ?) the case that RESET has highest priority (it may be unmaskable.)
>
> 2) There's an important distinction to be made between code that
> can't be safely interrupted (presumably because it's doing some
> transaction-like operation and that transaction has to complete
> atomically and can't be safely reentered) and code that can't be
> safely aborted (can't be interrupted in a way that could cause
> a non-local exit to throw past the point of resumption.)  Assuming
> that CLOSE is reentrant, it's probably true that it's safe to
> interrupt WITH-OPEN-FILE's cleanup form but undesirable to abort past
> it.  (Unless, of course, it isn't.)

This distinction is there in sbcl's without-interrupts and the proposed 
without-asynchronous-unwind.

About reentrancy: even if CLOSE is not reentrant we can just say so and 
ignore the problem or make CLOSE reentrant if it's sufficiently useful. 
There are a lot of things that cannot be (or simply are not) reentrant. 
Ideally, what's safe to call from an interruption is documented.

But unwinding from interruption must be allowed. Well, at least it seems 
too damn useful to me to disallow them. WITH-TIMEOUT (whose existence 
may very well be a fundamental mistake) is made easier to implement, 
and C-c then (ABORT) makes an even more convincing case for it. So, the 
distinction is fine, it's there, but how to protect normal code from 
async unwinds best is still unclear.

> You can -almost- firewall cleanup forms yourself (ignoring
> asynchronous interrupts) via slightly twisted constructs like:
>
> ;;; this is WITH-OPEN-FILE's cleanup form, hypothetically.
>
> (let* ((closed nil))
>    (loop
>     (unwind-protect
>       (progn
>         (close file)
>         (setq closed t))
>       (go CHECK))
>     CHECK (if closed (return))))
>
> but (with good reason) no one would want to write code like that.  It
> might be possible to write macros that wrote code like that, and that
> might be useful functionality (of possibly limited applicability.)
>
> This sort of firewalling is sometimes useful in the case of
> callbacks; it may be hard to know how stateful the foreign code that
> called you is, and it may be hard to understand the consequences of
> throwing/ aborting past that (possibly stateful) foreign code.

I see. It's indeed useful. Under the proposed solution it would be 

  (without-asynchronous-unwind ()
    (call-foreign-code ...))

with the extra requirement that callbacks have to be called via 
INVOKE-INTERRUPTION that sets up the unwind barrier.

In the patch all interruptions are invoked via that function (there is 
also a convenience macro WITH-INTERRUPTION). However, this use case 
suggests that things should be renamed because asynchronity is not the 
only source of unwind unsafety:

WITHOUT-ASYNCHRONOUS-UNWIND => WITH-UNSAFE-UNWIND

INVOKE-INTERRUPTION => INVOKE-WITH-UNSAFE-UNWIND-BARRIER
WITH-INTERRUPTION => WITH-UNSAFE-UNWIND-BARRIER

> I'm not sure that either of these points leads anywhere. I think that
> a first reaction that says "the right way for some of these things to
> interact is to disable/defer interrupts during cleanup" probably
> offers almost as many ways to lose as the status quo and I don't know
> that it makes the model simpler.  In most environments, doing
> arbitrary things "at interrupt time" is justifiably seen as unwise;
> it's not clear that effort expended towards guarding against possibly
> unwise actions is well-spent.  (But I confess that I sometimes
> believe otherwise ...)

Unwinding from interruptions and callbacks is very useful in a 
controlled manner. I still consider it a bargain, because the 
complexity is not all that high.

About the priorities, I'll have to ponder what they could bring to the 
table. At the moment I may be too fixated on asynchronous (should I say 
unsafe) unwinds to think clearly.

Cheers, Gábor