[Openmcl-devel] ANSI Lisp Tests

[Gary Byers]

On Thu, 2 Nov 2006, Brent Fulgham wrote:

> I ran the Paul Dietz's ANSI Lisp suite against OpenMCL 1.1 (32 and 64-
> bit).  (You can get the test suite by grabbing the source tarball
> from the GCL site)

I think that the tarball is probably a few years old; the test suite
in CVS contains a lot more tests (and generates a lot more failures.)

>
>
> 32-Bit OpenMCL 1.1
> ================
>
> Fails during MAKE-ARRAY.27:
>

It looks like it's actually failing on whatever follows MAKE-ARRAY.27;
the test name is generally only printed after successful completion.

>  MAKE-ARRAY.23 MAKE-ARRAY.24 MAKE-ARRAY.25 MAKE-ARRAY.26 MAKE-ARRAY.27
>
>
> 64-Bit OpenMCL 1.1
> ================
> CHAR-GREATERP.ORDER.3 CHAR-GREATERP.ORDER.4 FEATURES.1 FEATURES.2
> FEATURES.3
> 2 out of 10697 total tests failed: GET-PROPERTIES.8, UPGRADED-ARRAY-
> ELEMENT-TYPE.NIL.1.
> Welcome to OpenMCL Version 1.1-pre-061024 (DarwinPPC64)!
> ?
>
> 1.  Test GET-PROPERTIES.8 failed
> Form: (GET-PROPERTIES '(1000000000000 B C D) (LIST (1+ 999999999999)))
> Expected values: NIL
>                  NIL
>                  NIL
> Actual values: 1000000000000
>                B
>                (1000000000000 B C D).
>

I'm pretty sure that this was removed from the test suite a year or
two ago.  Quick quiz: is (1+ 999999999999) EQ to 1000000000000 ?

a) yes
b) no
c) that depends entirely on how an implementation represents FIXNUMs
    and on whether or not 1000000000000 is a FIXNUM.

Since (c) is the correct answer, the test is bogus.

> 2.  Test UPGRADED-ARRAY-ELEMENT-TYPE.NIL.1 failed
> Form: (LET ((UAET-NIL (UPGRADED-ARRAY-ELEMENT-TYPE NIL))) (LOOP FOR E
> IN *UNIVERSE* WHEN (TYPEP E UAET-NIL) COLLECT E))
> Expected value: NIL

There are several differences of opinion on what (if anything) the
UPGRADED-ARRAY-ELEMENT-TYPE of NIL is.  This is controversial enough
that newer versions of the GCL test suite have a switch which causes
tests which depend on a particular interpretation of that to be
skipped.