[Openmcl-devel] buffering for socket streams

[Gary Byers]

On Mon, 11 Nov 2002, Timothy Moore wrote:

> Hello,
> I'm doing my McCLIM work using openmcl and John Wiseman's CLX port.
> Performance is visibly bad.  I had blamed XDarwin, but running a fairly
> complicated remote application persuaded me that this is not the case.
> Instead I think I can blame suboptimal socket/stream options.  I added
> the :nodelay option to the make-socket call, which has helped quite a
> bit, especially with "control" operations like creating windows.
> However, there's still some visible slowness which I suspect is caused
> by buffering, or the lack of it.  I'd like the stream to be fully
> buffered (CLX will empty it when it's accumulated one or several
> protocol requests).  How do I do that?
>
> Thanks,
> Tim
>
>

All (well, practically all) streams in OpenMCL are buffered (in the
sense that lisp will attempt to I/O to and from them in chunks of
a few KB - by default - at a time.)  MAKE-SOCKET doesn't seem to
allow you to specify the size of the resulting stream's buffer
(which defaults to the value of CCL:*ELEMENTS-PER-BUFFER* at the
time of the MAKE-SOCKET call.)  Presumably it should do so; the
default value of CCL:*ELEMENTS-PER-BUFFER* (2K) may be inappropriate
for socket streams and/or for the type of I/O that CLX does.

All (well, practically all) streams in OpenMCL are Gray streams, so
functions like READ-CHAR and WRITE-BYTE involve some generic function
dispatch.  That's sometimes a source of overhead; I'm not sure if that
applies here.

There's also buffering going on inside the TCP stack; the :NODELAY
option (and other things) affect that.

Aside from the GF dispatch overhead involved in element-at-a-time
calls, I don't think that there's a lot of gross inefficiency in the
stream code per se (essentially the same code is used for sockets,
files, pipes, ttys, ...).  A thread that's unable to do I/O will
PROCESS-WAIT until it can; if other threads can run while it's
waiting, there's some potential latency there.

Are you running Jaguar ?  If so, is /usr/bin/ktrace installed ?
(I don't remember whether this comes with the OS or if you have
to install Apple's developer tools to get it.)

Is the client talking to the server via the loopback interface
(localhost/127.0.0.1), or via a regular IP address ?  (In either
case, I wonder if there's any way to sniff local traffic.)

Either ktrace or some sort of packet sniffer might give us an
idea of how much time we're spending in the OS (and/or re-trying
or waiting to do I/O) vs time spent in other things.

I've been meaning to ask John Wiseman where to find his CLX port
for .. oh, however many months it's been since he announced it.
Maybe I should try to find it and see what ktrace/sniffing/poking
around and guessing reveal.

If I had to guess, I'd say that we're "waiting too long for I/O
to complete" rather than "not buffering enough and therefore making
too many I/O calls", but a guess based on some packet/system call
traces would be a more informed guess ...



> _______________________________________________
> Openmcl-devel mailing list
> Openmcl-devel at clozure.com
> http://clozure.com/cgi-bin/mailman/listinfo/openmcl-devel
>
>


_______________________________________________
Openmcl-devel mailing list
Openmcl-devel at clozure.com
http://clozure.com/cgi-bin/mailman/listinfo/openmcl-devel