[Openmcl-devel] string-output-stream thread unsafety with hunchentoot (ccl, darwin/x86-64)

[Gary Byers]

STRING-OUTPUT-STREAMS are themselves tiny and cheap to create (and are
usually fairly short-lived and are assumed to be thread-private), but
they contain structure instances (STRING-OUTPUT-STREAM-IOBLOCKs) that
can be fairly large and relatively expensive to create and initialize.
When a STRING-OUTPUT-STREAM is closed, its ioblock structure is
basically pushed on a freelist; when a STRING-OUTPUT-STREAM is
created, its ioblock structure is popped from that freelist if
possible.  The freelist is assumed to be thread-private, and the code
that does the pushing and popping (recycling) does so with interrupts
disabled.  The two functions that deal with this -
%CLOSE-STRING-OUTPUT-STREAM and CREATE-STRING-OUTPUT-STREAM-IOBLOCK,
both defined in ccl/level-1/l1-streams.lisp - are actually fairly
simple (modulo the details), and if there's a way for things to go
wrong when all of these assumptions are met it's not obvious to me.

The assumptions in question are that:
  - string-output-streams are thread-private
  - each thread has its own binding of CCL::%STRING-OUTPUT-STREAM-IOBLOCKS%
    (whose value is the data structure used as a freelist.)

Doing output to a single STRING-OUTPUT-STREAM from multiple threads -
unless some higher-level locking protocol is used - risks all kinds
of lossage. Closing a string output stream from multiple threads
could lead to the the symptom shown in the backtrace (I guess that
additional assumptions are that a closed ioblock is on at most one
thread's freelist and that no ioblock associated with an open stream
is on any thread's freelist.  An active/open ioblock's STREAM slot
points back to the stream that uses it; an inactive/closed ioblock's
STREAM slot points to the next entry on the freelist.  The error
is a complaint that a STREAM object was found on the freelist, which
could easily happen if either of these most recent assumptions were
violated.)

I'm fairly sure that the same symptom could happen if threads shared
a (static) binding of CCL::%STRING-OUTPUT-STREAM-IOBLOCKS%, but so
many other bad rhings could happen in that case it's not clear why
this particular bad thing would happen more consistently than any
other.

I don't think that any of the assumptions above are too unreasonable,
but I'd also agree that it should be possible to reliably use a
string output stream from multiple threads (in the exceptional case
where it makes sense to do so intentionally) as long as some sort
of external locking protocol is used.  E.g.

(defclass shared-string-output-stream (fundamental-character-output-stream)
   "or whatever the appropriate Gray stream class is"
  ((real-stream :initform (make-string-output-stream) ...)
   (lock (make-lock))))

(defmethod stream-write-char ((stream shared-string-output-stream) char)
   (with-slots (real-stream lock) stream
     (with-lock-grabbed (lock)
       (write-char real-stream char))))

etc., and don't forget:

(defmethod close ((stream shared-string-output-stream) &key abort)
   (declare (ignorable abort))
   (with-slots (real-stream lock) stream
     (with-lock-grabbed (lock)
       (close real-stream))))

It's also possible (and very, very occasionally useful) to create
a thread that doesn't use standard initial bindings (including
the binding of CCL::%STRING-OUTPUT-STREAM-IOBLOCKS%.)  That thread
basically wants to run in a very constrained and controlled environment.
(I suppose that we could check to see if the calling thread had a
dynamic binding of the freelist variable and refuse to use the 
freelist if so, but unless/until that happens a constraint on the
environment in which such a thread runs is that the thread not use
STRING-OUTPUT-STREAMs.)



On Thu, 9 Apr 2009, Andreas Fuchs wrote:

> Hi there,
>
> I've just spent a few hours chasing what seems to be a thread unsafety
> issue with ccl on darwin (I'm running on a x86-64 machine) and
> hunchentoot 1.0: at least this is where it's easiest to reproduce.
> There's a reduced test case (may even count as small if you discount
> hunchentoot and its deps) at <http://paste.lisp.org/display/78304#2>.
> I've tested with Clozure CL version 1.3-dev-r11918 and 1.3-dev-r11912.
>
> I am seeing breakage like the backtrace pasted above the test case;
> other nasty things have been happening, too, like ccl terminating with
> a Bus Error when it's running in slime.
>
> According to Hans Hübner on #lisp, this isn't reproducible on
> FreeBSD/x86, so may be a darwin-related issue.
>
> Please let me know if there is any more information you need to debug this,
> -- 
> Andreas Fuchs, (http://|im:asf@|mailto:asf@)boinkor.net, antifuchs
> _______________________________________________
> Openmcl-devel mailing list
> Openmcl-devel at clozure.com
> http://clozure.com/mailman/listinfo/openmcl-devel
>
>