[Openmcl-devel] Setting time out value when creating a connected stream socket with MAKE-SOCKET

[Gary Byers]

On Mon, 31 Dec 2007, Erik Huelsmann wrote:

> On 12/26/07, Gary Byers <gb at clozure.com> wrote:
>
> Thanks for your long reply. From your reply I learned - again - that
> its hard to be concise when posing a question. I'll try to be more
> exact in this mail.
>
>> Way back when (when threads were cooperatively scheduled), all
>> socketes were created non-blocking; you couldn't block indefinitely
>> waiting for network I/O without locking out the lisp scheduler,
>> so there was code that handled the EAGAIN/EWOULDBLOCK error that's
>> returned when an I/O operation on a non-blocking socket can't
>> complete immediately, and the lisp scheduler would do periodic
>> short-lived #_select calls to try to determine which thread's
>> sockets were ready to do I/O and which threads should therefore
>> get activated.
>
> There are actually 3 'types' of time outs:
>
> 1) When calling connect()
> 2) When calling read()
> 3) When calling write()
>
> Your answer addresses the last 2. The requestee on the Drakma mailing
> list implemented behaviour for the last 2 too. My question however was
> about the first item.
>
> I wasn't asking about - although I'm interested in - the last 2
> because I'm working on a select() like interface for item 2 (hopefully
> followed by something which also supports item 3). [I try to take
> small steps at a time.] There are a lot of problems associated with
> using non-blocking sockets the way usocket currently works. This is
> why select() can be made to work for read() - given blocking sockets.
> Being able to specify a timeout before a certain minimum amount of
> data has been received (using SO_RCVTIMEO/SO_SNDTIMEO) is very
> attractive, but a bit too complicated for me to handle at this time.
> I'll come back to that at a later stage.
>
> It doesn't seem possible - without the cooperation of the
> implementation - to limit the time out value for connect(). Several
> implementations do provide the facilities to specify the time out on
> connect (Allegro provides WITH-TIMEOUT, CLISP a :TIMEOUT key
> argument).
>
>> That's all (thankfully) gone, but there's still some low-level
>> I/O code that basically says "it'd be better to wait forever
>> in #_select than to wait forever in #_read",
>
> Right. Luckily, I wasn't trying to get a lisp-side thread-scheduler
> implemented, rather I'm trying to give applications control over how
> long a thread blocks on certain calls.
>
>> Using non-blocking sockets and a #_select timeout was the traditional way of doing this,but it doesn't always work a whole lot better in C programs
>> than it would in Lisp programs.
>
> True, but even though the interface may be 'clumsy' or non-intuitive
> to beginning network programmers, it has worked 'perfectly' for a long
> time for many network related programs. The key here is - I believe -
> to leave handling of the waiting part to the calling application: that
> way, it can decide to do "other stuff" such as processing non-network
> related signals etc. Doing all that correctly is - however - not
> trivial.

My strong impression ("strong" doesn't always mean "correct", of course)
is that this approach doesn't scale well; you wind up doing something
like a lisp-side scheduler, and it may be hard to do a better job of
scheduling things than the OS is already doing.

>
>> Modern Unices define socket options (#$SO_RCVTIMEO and
>> #$SO_SNDTIMEO) that cause reads and writes that block longer
>> than the specified time to return EAGAIN/EWOULDBLOCK.  So:
>
> Yes, but the Unix Sockets FAQ (see
> http://www.developerweb.net/forum/archive/index.php/t-3439.html)
> suggests that the way they are implemented differs greatly among the
> implementing OSes, meaning that it's very hard to make use of them in
> portable code. The easiest way to create portable code would then be
> using non-blocking sockets and select().

It might be interesting to see what (for instance) Apache (>= 2.0)
does (assuming that it wants to enforce I/O timeouts in some
circumstances and that it has to deal with lots of portability
issues.)

>
>> - If we only care about setting timeout options on socket creation,
>> it'd be fairly straightforward to add :READ-TIMEOUT and :WRITE-TIMEOUT
>> options to MAKE-SOCKET and arrange that those options set the
>> appropriate socket options.  (It'd be harder to set these options
>> on the fly, though that might be possible.)
>
> Looking at LispWorks, they provide the read/write timeout at socket
> creation time, meaning that I won't get much further with usocket than
> that. If you were to implement this 'solely' for the purpose of
> usocket consumers, it'd be fine if it were handled at socket creation
> time.
>
>> - any current code in CCL that handles EAGAIN as a transient
>> condition on a non-blocking fd should be changed to signal
>> a timeout error if the fd in question is blocking.  (If the
>> fd in question is non-blocking, they may want to ask how
>> it got that way ...)
>
> Hmm. That would mean a complete screening of all uses of raw FDs in
> all stream code? Sounds like quite a task.

Why would that be true ?

If a low-level read call (for instance) returns the error EAGAIN (aka
EWOULDBLOCK) that presumably means one of two things:

a) the fd involved in the read call is non-blocking and the read
would have blocked; code can block in select, busy wait ...

b) the fd involved in the read call is blocking and a timeout
was exceeded, and (in the case of read or other data transfer
operations) that's an error.

That means that in the relatively few places that actually
read (for instance) from a file descriptor (all of which
currently handle EAGAIN by blocking indefinitely in select())
code would would have to check the non-blocking flag on
the socket in order to differentiate between those cases.

I think that I'd want to vote against having non-blocking connected
sockets; I'd be interested in hearing if people think there are good
reasons for having them if timeouts are otherwise supported.  One
advantage of not supporting non-blocking connected sockets is that
that eliminates case (a) above.

>
>> Would that do it (assuming that #$SO_RCVTIMEO and #$SO_SNDTIMEO
>> are implemented on all interesting platforms ?)
>
> Yes, that would, except that it doesn't address the concern I tried to
> get addressed primarily (for now): limited timeout on connect() calls.
> Would it be simpler to address timeout on connect()? If so, would it
> not be better we handle that first?

I've been working on the send/receieve timeout stuff; so far, I
haven't seen a lot of differences between different OS's behavior
(though I confess that I haven't tried it on Darwin yet.)  I -have-
seen that this stuff didn't behave the way I'd remembered ...

For connect timeouts, I don't know of another way to do it besides:

  - set non-blocking mode on the socket
  - try to connect; if that returns EAGAIN, select with the specified
    timeout
  - if/when we successfully connect, strongly consider the idea of
    making the socket blocking.

That would all mean that MAKE-SOCKET would be changed to accept new
:CONNECT-TIMEOUT, :RECEIVE-TIMEOUT, and SEND-TIMOUT arguments.  So
far, it seems that the latter two can be made to work reliably
and portably via the #$SO_*TIMEO socket options, but it's probably
more important that they work in some way than that they work in
some particular way.

>
> bye,
>
>
> Erik.
>
>
>> On Wed, 26 Dec 2007, Erik Huelsmann wrote:
>>
>>> Hi,
>>>
>>>
>>> I'm still the maintainer of usocket [a cross Lisp, cross OS portable
>>> sockets abstraction layer/portability layer]. I've been kindly
>>> requested to extend the functionality of usocket with a user
>>> configurable time out period on a connected stream socket.
>>> Unfortunately, I've been unable to determine how I should achieve that
>>> with CCL. Since you have chosen to provide nearly the same interface
>>> as Franz, this is how Franz does it:
>>>
>>> (sys:with-timeout (<time-out-value> :connection-timed-out)
>>>  (make-socket :type :stream :connect :active :host
>>> "www.common-lisp.net" :port 80))
>>>
>>> which returns :connection-timed-out upon timeout.
>>>
>>> SBCL and CMUCL will allow me to implement the above using non-blocking
>>> sockets, but it looks like CCL won't let me create non-blocking
>>> sockets either...
>>>
>>> Could you give me any advice or pointers on how to proceed?
>>>
>>>
>>> Thanks in advance!
>>>
>>> bye,
>>>
>>>
>>> Erik.
>>> http://www.common-lisp.net/project/usocket/
>>> _______________________________________________
>>> Openmcl-devel mailing list
>>> Openmcl-devel at clozure.com
>>> http://clozure.com/mailman/listinfo/openmcl-devel
>>>
>>>
>>
>
>