[Openmcl-devel] slow read-char

[Erik Pearson]

Hi Takehiko -- you are not alone!

I recently experienced performance problems using READ (and probably  
READ-BYTE) -- you can find the messages on the list from around June  
21-25. My findings with Shark were similar to yours. When you look at  
Shark's hierarchical view (don't remember what it is called) a lot of  
the activity is underneath READ, where I saw the locking going on. In  
my case it was some very high figure like 60% of activity associated  
with READ.

I figured it had something to do with making streams work with  
multiple thread access, but didn't really pursue it further (rather  
bookmarking it for future exploration, but I've got so much  
bookmarked material that I'm really glad you encountered the same  
issue and brought it up!)

My solution was to implement file caching in order to reduce the  
amount of file reading that needs to happen during the live of the  
application. This was in a web application which had run fine with a  
lot of file reading on Allegro CL, but then slowed down a lot in  
Openmcl. I went ahead and implemented file read and write caching and  
performance immediately improved. In this case, these were changes  
that I had planned on making one day if performance became in issue,  
and work well in this context.

Of course, it would still be a treat to see READing be faster. The  
proposed solution seems sensible -- streams could be marked for  
operation in different contexts -- shared  streams would need to be  
controlled as currently, but private streams could only be accessed  
by a single thread at a time, with the ability, of course, to control  
which thread has access the the stream.

It is possible that I'll run up against this same issue with reading  
and writing network connections -- I haven't Sharked this yet.

I suppose it is possible that there are some more hackish methods,  
like building non-locking (and non-CL) read/write primitives that can  
at least work with unix file descriptors and bytes (which is all I  
need)??

BTW It may not be fair to compare Openmcl and Allegro CL in READ  
performance, since Allegro is not native-threads on OS X.

Erik.


On Jul 14, 2006, at 7:39 AM, Takehiko Abe wrote:

> Gary Byers wrote:
>
>> It's also worth checking to ensure that the locking is really
>> where the problem is.  That seems likely, but it'd probably
>> be a good idea to run your test case through CHUD/Shark.
>
> I run the chud-metering.lisp for the first time. Still don't know
> how to read the result properly, but ccl::%unlock-recursive-lock
> (8.0%), SPmkunwind (7.4%) and ccl::%lock-recursive-lock (6.8%) are
> listed on the top.
>
> ***
>
>>
>> The simplest way of avoiding it that comes to mind is to introduce
>> thread-private streams.  I suspect (and this may vary a bit from
>> person to person) that most application-level streams are only
>> accessed from a single thread, and it might therefore be appropriate
>> to make "thread-private" streams be the default.
>
> I think the change would be nice -- I think it will be a boost
> for some applications... But, nobody has mentioned that read-char
> is slow before. So I'm not sure.
>
> On the other hand, I guess that if one wants to access a stream
> from multiple threads one has to lock it anyway regardless openmcl
> does its own locking or not.
>
>
> Thanks,
> T.
>
>
> _______________________________________________
> Openmcl-devel mailing list
> Openmcl-devel at clozure.com
> http://clozure.com/mailman/listinfo/openmcl-devel