[Openmcl-devel] Fun With Data: Historical Benchmarks

Brent Fulgham bfulg at pacbell.net
Tue Oct 17 20:35:10 UTC 2006


Hi Andrew:

For this benchmark run, 'Reference' == 'OpenMCL'.  The benchmark clarifies this at the bottom of the page, but I just added a line at the top of the page to make this clear.

I think the page is immutable if you are not logged in to the wiki.

Thanks,

-Brent

----- Original Message ----
From: Andrew Shalit <alms at clozure.com>
To: Brent Fulgham <bfulg at pacbell.net>
Cc: Gary Byers <gb at clozure.com>; OpenMCL <openmcl-devel at clozure.com>
Sent: Tuesday, October 17, 2006 8:12:38 AM
Subject: Re: [Openmcl-devel] Fun With Data:  Historical Benchmarks

Brent --- all I see on that page is columns for "Reference" and "SBCL".  
Is the second column actually OpenMCL?

Also, why is the wiki page immutable?

Andrew

Brent Fulgham wrote:
> On Oct 16, 2006, at 5:42 PM, Gary Byers wrote:
>
>   
>> Sorry for not responding to your original message.
>>
>> One of the things that stuff like this (tracking changes to  
>> compliance, tracking
>> performance between OpenMCL releases) more meaningful and useful is  
>> if those
>> releases is if those releases were more frequent; it's been over a  
>> year since
>> 1.0, 1.1 is nearing feature-completness (some Unicode support!) but  
>> will need
>> some testing, and that's pretty bad.
>>     
>
> Well, I guess you just need some help!
>
>   
>> I'm sure that this'd be useful (thanks), but it sort of goes without
>> saying that benchmarking's a black art; I'd tend to trust results
>> that're consistently bad (like I/O in 1.0) more than those that appear
>> good.  I was just timing something a few minutes ago, and tended to
>> get better results when I introduced code that should have slowed
>> things down slightly.
>>     
>
> I posted the results of my quick run of the benchmark (http:// 
> openmcl.org/openmcl-wiki/HowFastAreWe#preview).  I'll run it again  
> with OpenMCL 1.1 soon to see where things stand...
>
>   
>> (Just the same, things that we know about - like I/O performance -  
>> may have gotten
>> gradually worse over time.  If that's true, timing results over  
>> time would have
>> shown that trend and might have caught the problem earlier, and  
>> having that
>> sort of thing set up may keep other things from drifting into the  
>> realm of the
>> very bad.)
>>     
>
> Sure.  We all know [Twain/Disraeli/?]'s famous quote.  But it's  
> mostly sudden changes that are interesting.  Even the funny tests on  
> http://shootout.alioth.debian.org/ create their own drama, etc.  But  
> it's useful to see trends, and sometimes highlights true problems.
>
> At any rate, it gives me an excuse to play with the compiler and  
> hopefully it provides some utility (even if only entertainment).  :-)
>
> Thanks,
>
> -Brent
> _______________________________________________
> Openmcl-devel mailing list
> Openmcl-devel at clozure.com
> http://clozure.com/mailman/listinfo/openmcl-devel
>   







More information about the Openmcl-devel mailing list