[Openmcl-devel] Fun with Measurements

[Brent Fulgham]

Hi everyone,

I re-ran the benchmarks with the current OpenMCL 1.1 snapshots, both  
in the standard 32-bit PPC and 64-bit PPC.

You can see the results on http://openmcl.org/openmcl-wiki/HowFastAreWe

I think the results are interesting; SBCL is substantially faster  
than any of the OpenMCL variants on several benchmarks, but OpenMCL  
is outstanding in a few others.

Some of the worst cases are the BIGNUM-related routines (see the  
scores on BIGNUM/ELEM-10000-1, BIGNUM/PARI-200-5, and PI-DECIMAL/BIG  
for some especially egregious cases).  SBCL also seems to have  
superior floating point support, as evidenced by the huge performance  
difference in MRG32K3A (a multiply-recursive random number  
generator) , FFT, and ?.

While the 64-bit build of OpenMCL looks great on nearly all tests  
(and substantially improves over the 32-bit build in some cases),  
something seems very wrong with the 64-bit support for BIGNUMS (see  
BIGNUM/PARI-200-5, PI-DECIMAL/BIG and PI-DECIMAL/SMALL for example).   
Remember that the non-reference columns are relative scales, so on PI- 
DECIMAL/SMALL, 64-bit OpenMCL is 10x as slo as OpenMCL 1.1/1.0 in 32- 
bit.  One small bright spot is the CRC40 benchmark, which due to it's  
heavy use of 40-bit integers is substantially improved using native  
64-bit words.

A very puzzling result is that of the various ARRAY benchmarks.   
While SBCL is extremely slow for the 1D case, it shows remarkable  
performance in the 2- and 3-D array tests.  Perhaps SBCL's internal  
representation of ARRAY is optimized for the more-common 2- and 3-D  
specializations, or perhaps the difference is due to SBCL's seemingly  
more efficient list implementation. (see also the TAKL and WALK-LIST/ 
SEQ performance).

Maybe someone smart could take a look at what might be causing such a  
huge bottleneck in the 64-bit BIGNUM implementation...

Thanks,

-Brent