[Openmcl-devel] thread pool question

[Gary Byers]

On Thu, 13 Jul 2006, Erik Pearson wrote:

> Hi Gary,
>
> I'm trying to trim down my web server app, and one area where I've
> focused is no thread usage. I've not done a real empirical test for a
> while, but since each connection to the server creates one to four
> new threads in keep-alive mode (it depends on the web browser, and
> I've just begun to notice this behavior when enabling keep-alive
> connections), or a rapid succession of new threads in non-keep-alive
> mode.
>
> In any case, I want to reduce any latency due to thread creation.
>
> I've implemented a crude thread pool, which I disabled temporarily to
> deal with the recent process-input-wait /  fd-input-available issues.
> Thread pooling did increase performance, although I'm not sure if was
> perceptible from the point of view of user.
>
> However, I'm wondering if you've tried to implement thread pooling
> yourself -- or anyone else on the list -- and what inherent issues
> might be lurking. I'm assuming there is no danger in keeping the os
> threads alive for as long as one likes -- but I'm not sure about the
> lisp resources associated with the thread that may need resetting
> between functions.

I did something in the #+openmcl-native-threads Portable Allegroserve 
a few years ago (I don't know if it's still in the distribution) where
the basic architecture involved a queue (lock, list, and semaphore)
and a basically fixed set of worker threads doing:

(loop
    (let* ((request (pop-request-from-queue)))
      (process-request request)))

A server thread would push requests onto the queue and would occasionally
create a new worker thread (up to some configurable limit) if it saw
the queue filling up (it may be hard to really monitor that accurately).

I know of someone else whose code uses a similar technique; for
something like this (all of the worker threads doing essentially the
same thing) it seems to work fairly well.

>
> My first attempt at getting this to work relied on preventing the
> threads from going dead (exhausted) by resetting them just before the
> thread's function exits (basically the thread runs an anonymous
> function whose job is to wrap the target function so that it doesn't
> exit with errors.). I think I found that exhausted threads could
> sometimes be reused and sometimes not, but the docs say not to do this.
>

A lisp PROCESS object will stick around until it gets GCed.  The
underlying OS-level thread (and some lisp glue in between) will
get deallocated when the thread's initial function exits; if you
are able to reset/preset/enable a PROCESS whose thread has died,
then all that you're really recycling is the PROCESS object (and
not the OS thread and its stacks.)

> Or perhaps I'm going about this wrong -- the thread could run a
> function whose job is to just wait (i.e. on a semaphore.) for a
> function to be provided to it (i.e. setting some variables or slots),
> then run that function, then when it is done re-enter the wait loop.
>

I think that that's what I'd recommend;  PRESET/RESET involve some
handshaking and synchronization.

> Do you know what part of the thread-creation process is in general
> the most time-consuming? OS thread creation? Lisp resource creation?
> Setting up the thread?

My intuition is that the stuff that happens on the OS/foreign side
(thread and stack creation) is more expensive than what happens on
the lisp side (MAKE-INSTANCE and some handshaking. I haven't tried
to meter this with Shark, so my intuition might be wrong.

Some of that overhead involves allocating a few MB for some extra
lisp stacks.  At various points in history, there have been attempts
to manage/recyle those stacks.  That isn't done currently, but I
could imagine that doing so could help in some cases.  (The cases
that it could help probably involve creating lots of short-lived
threads, and my intuition suggests that that's a bad idea that
can probably be avoided a lot of the time.)

>
> Thanks,
> Erik.
> _______________________________________________
> Openmcl-devel mailing list
> Openmcl-devel at clozure.com
> http://clozure.com/mailman/listinfo/openmcl-devel
>
>