[Openmcl-devel] buffering for socket streams

[Gary Byers]

On Mon, 11 Nov 2002, Timothy Moore wrote:

> Thanks for the information.  What effect does :interactive have on
> ioblock-streams?

Not too much: if data isn't immediately available when reading from
an "interactive" stream, the thread will go into a select/process-wait
state until it thinks that it can read without blocking.  File streams
are actually about the only type of stream that isn't interactive in
this sense.

The read-eval-print loop mistakenly assumes that EOF from an
"interactive" stream is a transient condition.  I believe that this is
actually only true of TTYs

>
> I don't blame any gross inefficiency in the streams code, or indeed any
> performance problem in openmcl itself.  I do suspect it's something
> like you suggest: waiting too long for I/O to complete.  I'm just doing
> a very unscientific comparison between CLX/CMUCL on a 1Ghz x86 Linux
> laptop and CLX/openmcl on a 700 Mhz iBook; the openmcl combo has
> visible pauses as each character is rendered on the screen.  Granted we
> shouldn't be writing one character at a time through X, but in the
> CMUCL case we seem to be able to get away with it.

The (CLOS) overhead that's related to how buffered streams are implented
might be measurable, but I doubt if it would really lead to visible
pauses.

>
> On Monday, November 11, 2002, at 10:21  PM, Gary Byers wrote:
>
> >
> > Is the client talking to the server via the loopback interface
> > (localhost/127.0.0.1), or via a regular IP address ?  (In either
> > case, I wonder if there's any way to sniff local traffic.)
> Yes.  I would like to get Unix domain sockets working too.

There might be a slight difference between using the loopback interface
and using a "real" (ethernet) interface's IP address.

I don't think that there's anything conceptually hard about implementing
Unix domain sockets, but I haven't really thought about it.

>
> > I've been meaning to ask John Wiseman where to find his CLX port
> > for .. oh, however many months it's been since he announced it.
> > Maybe I should try to find it and see what ktrace/sniffing/poking
> > around and guessing reveal.
> There are some notes in the McCLIM sources on getting and building CLX
> for openmcl.  I think we have all the relevent patch files in the
> sources too.

I had actually done a port of a slightly different version of CLX to
OpenMCL a year or so ago, but had forgotten where I'd put it.  It's
been on clozure.com's FTP server (of all places):

ftp://clozure.com/pub/CLX

Hmm.  Apparently, it was exactly a year ago Monday ...

I remember running a few of the CLX demos and don't recall thinking
that there was a performance problem; OTOH, this would have been under
Linux.  We don't yet know what the problem is, but it's -possible-
that differences between OSX and Linux could explain some of it.

> >
> > If I had to guess, I'd say that we're "waiting too long for I/O
> > to complete" rather than "not buffering enough and therefore making
> > too many I/O calls", but a guess based on some packet/system call
> > traces would be a more informed guess ...
> Is there any way to tune that waiting?  Or is long wait the result of a
> context switch?

Depending on what's causing the waiting, we might be able to tune it.
There are typically a lot of tunable parameters related to how buffering
takes place inside the OS kernel (the :NODELAY option to MAKE-SOCKET
affects this, in at least one direction.)  Note that getting data from
one end of the connection to the other is a fairly complicated process,
even when both ends of the connection are running on the same machine:
both (OS) processes and the kernel need some CPU time, and the kernel
has to make policy decisions (e.g., is it better to wake up a blocked
reader as soon as a little data arrives, or would it be more efficient
to wait for more data ?  "Better" may depend on whether low latency
or high throughput is "better".)

It's also possible that the lisp stream code could be tuned a bit,
and/or that the lisp's buffer size isn't a good match for the buffer
size used by the OS for the socket.

The ktrace program I mentioned earlier can be used to trace system
call activity; it might give us an idea of how I/O's being done and
of how long it's taking to complete.  That might not tell us exactly
what needs to be tuned, but could tell us whether it's worth looking
at socket options or whether it'd be better to look elsewhere.

I'll try to get some version of CLX working and try to do what you're
doing under ktrace.

Is it possible/practical to use the TIME macro around a CLX-WRITE-CHAR
loop ?  If so, do the times reported for system/user time vs wall-clock
time reveal anything interesting ?

>
> Tim
>
>
>


_______________________________________________
Openmcl-devel mailing list
Openmcl-devel at clozure.com
http://clozure.com/cgi-bin/mailman/listinfo/openmcl-devel