[Openmcl-devel] slow read-char

[Gail Zacharias]

At 7/20/2006 02:47 PM, Gary Byers wrote:
>On Thu, 20 Jul 2006, Gail Zacharias wrote:
>>At 7/20/2006 06:15 AM, Gary Byers wrote:
>>>The good news is that I was able to get about a 3X speedup of a test
>>>case that READ-CHARs 2M characters from a file by making things like
>>>OPEN and MAKE-SOCKET accept a :PRIVATE argument (which effectively
>>>makes the resulting stream permanently locked by the calling thread
>>>and eliminates the per-character locking overhead);
>>
>>Does it actually prevent the stream from being used by other threads?
>>It would be useful to have streams that are usable in multiple threads
>>but just assume that locking happens at a higher level.
>
>I had thought that - assuming that there's a cheap way of "transferring
>ownership" of a stream - that any sort of higher-level locking protocol
>would/could just do that transfer as a fairly small step. but I'm
>not sure that that's true.  (It might be a small step, but it might
>also be an awkward one; once a thread has won the right to access the
>stream via that higher-level protocol, it might be difficult for it to
>try to cajole the previous owner into giving up some additional token.)

Ahh, I hadn't thought of that.  Yes, having a way to transfer ownership
of a private stream would definitely be enough, if, as you say, it's
possible to do so without deadlock.

FWIW I don't think it has to be all that cheap - worst it only has to be
cheaper than 2xN where N is the number of I/O operations I'm going
to do between switches...

I think in practice, a "higher-level protocol" usually involves
some kind of dynamic-scope transactions, so just providing something
along the lines of:

    (WITH-EXCLUSIVE-STREAM-ACCESS (stream)
       (do-a-bunch-of-io-without-locking-overhead stream))

would do it, however it's implemented. Perhaps it could just lock whatever
lock is used by READ-BYTE, then even the naive READ-BYTE wouldn't be
incurring all the locking overhead (I'm assuming asking for a lock you
already have is fast).


>So yes, you should have enough rope to be able to create a stream that 
>depends on external locking mechanisms.  (I think that it's
>also desirable to distinguish between being persistenly locked/owned
>for reading and for writing in the case of an I/O stream, and last
>night's checkin doesn't address that yet, either.)
>
>So there are at least three cases that we've identified so far:
>
>1) something that uses ad-hoc safe-but-slow locking around every
>    transaction, i.e., the status quo.
>
>2) something which in practice will only be used by one (or in
>    the I/O case, two) thread(s) and for which the concept of
>    long-term or permanant ownership makes sense as a simple
>    exclusion mechanism/sanity check.
>
>3) something which uses external exclusion mechanisms (possibly
>    including luck and/or abstinence)
>
>Right now, every transaction decides (at runtime) whether case (2) or
>(1) applies.  That's kind of silly, and it'd likely be better to
>decide which case applies at stream-creation time and arrange to have
>the stream's I/O primitives handle that case only.  I can't think
>of a case where it'd be desirable for that to change over the lifetime
>of a stream.