[Openmcl-devel] type specifier '(simple-vector n) in defmethod

[Nicolas Martyanoff]

Ron Garret <ron at flownet.com> writes:

>> On Jan 6, 2024, at 6:00 AM, Nicolas Martyanoff <nicolas at n16f.net> wrote:
>> 
>> In CL, this really does not map well to streams because streams are
>> fundamentally blocking (see for example the semantic of READ-SEQUENCE).
>
> That's not true. CL:READ-SEQUENCE is blocking, but LISTEN works on
> binary streams so implementing a non-blocking version is an elementary
> exercise.

A function similar to LISTEN would not help in any way.

OS primitives dealing with non-blocking IO are event based: you do not
ask if you can read (or write) a file descriptor, you register it
against a some kind of multiplexer (e.g. an epoll instance), then
another primitive lets you ask this multiplexer which events were
signaled for any registered file descriptor. You then dispatch these
events manually which usually mean running the appropriate callback.

The approach is fundamentally top-down: you start from IO events, then
dispatch. But a read event only means you can perform one single
non-blocking read call. So you need to guarantee that any stream
function reading your file descriptor will perform one read and one read
only (if you do a second one, you'll potentially block with all the
infortunate consequences for your event loop). You cannot guarantee that
(heck you cannot even know if standard streams are buffered or not, and
don't get me started on external formats), and various stream operations
are fundamentally based on the idea of reading repeatedly
(READ-SEQUENCE, READ-LINE, etc.). So you are forced to bypass streams,
and use your FFI to call read() on your own and implement buffering
yourself. Similar issue when writing.

I work around this problem by implementing my own TCP streams using the
(non-standard) Gray stream API. In non-blocking mode, I signal a
condition indicating that I could not complete the operation because
doing so would require additional calls to read(). This way the IO mux
knows which events must be watched for. But then we're back writing all
parsers as operating on IO buffers instead of just reading streams, and
you still end up juggling with multiple callbacks for processing which
results in much more complex code.

These problems do not exist in Go or in Erlang: you can use a
stream-like system and just read the file descriptor. The runtime
handles IO in a non-blocking way and assigns another green thread for
execution until the read call can continue; when it happens, it schedule
the initial green thread for execution, and the read call returns. You
simply cannot emulate that in CL unless your implementation provides the
correct runtime support.

-- 
Nicolas Martyanoff
https://n16f.net
nicolas at n16f.net