[Openmcl-devel] thread pool question

[Erik Pearson]

Hi Gary,

I'm trying to trim down my web server app, and one area where I've  
focused is no thread usage. I've not done a real empirical test for a  
while, but since each connection to the server creates one to four  
new threads in keep-alive mode (it depends on the web browser, and  
I've just begun to notice this behavior when enabling keep-alive  
connections), or a rapid succession of new threads in non-keep-alive  
mode.

In any case, I want to reduce any latency due to thread creation.

I've implemented a crude thread pool, which I disabled temporarily to  
deal with the recent process-input-wait /  fd-input-available issues.  
Thread pooling did increase performance, although I'm not sure if was  
perceptible from the point of view of user.

However, I'm wondering if you've tried to implement thread pooling  
yourself -- or anyone else on the list -- and what inherent issues  
might be lurking. I'm assuming there is no danger in keeping the os  
threads alive for as long as one likes -- but I'm not sure about the  
lisp resources associated with the thread that may need resetting  
between functions.

My first attempt at getting this to work relied on preventing the  
threads from going dead (exhausted) by resetting them just before the  
thread's function exits (basically the thread runs an anonymous  
function whose job is to wrap the target function so that it doesn't  
exit with errors.). I think I found that exhausted threads could  
sometimes be reused and sometimes not, but the docs say not to do this.

Or perhaps I'm going about this wrong -- the thread could run a  
function whose job is to just wait (i.e. on a semaphore.) for a  
function to be provided to it (i.e. setting some variables or slots),  
then run that function, then when it is done re-enter the wait loop.

Do you know what part of the thread-creation process is in general  
the most time-consuming? OS thread creation? Lisp resource creation?  
Setting up the thread?

Thanks,
Erik.