[Openmcl-devel] Floating point exceptions on Windows

[Gary Byers]

On Tue, 5 May 2009, Gary Byers wrote:

[ramblings]

After R'ing TFM and spending a little quality time with GDB, I think
that I understand what's going on here (and, FWIW, it doesn't seem to
be Windows-specific.)  Some things that my earlier reply was confused
about:

- FF-CALL doesn't check to see if FP exceptions happened during the
call; what it generally tries to do is store the FP exception status
in thread-local storage after the call and ensure that the call is
entered with the FP exception status zeroed.  The code that does this
originally assumed that SSE2 was used for all FP math; that's true on
Darwin and on x86-64, but is not true on non-Darwin x86-32 platforms.
If we're running on a platform where we can't assume that SSE2 is used
for math library routines (the constant SSE2_MATH_LIB is undefined), we
store the x87 status word (rather than the SSE2 mxcsr) in thread-local
memory; we never do anything to clear the x87 status word.

We assume that the foreign function is executed with all FP exceptions
"masked" (e.g., marked in the control register to not cause a trap when
they occur; we save and restore the thread's lisp FP exception mask around
the FF-CALL.

On return from a foreign function call to a math library routine (used to
implement a CL math function), we check to see if any exception bits (in
the thread-local-storage word) are set that correspond to unmasked exceptions
in the thread's lisp FP exception mask; this is done using some fairly
ugly undocumented internal functions.  If an exception that would have caused
a trap if it were unmasked occurred, we signal a lisp error.

- x87 FP exceptions are indeed imprecise, but what this means is that the trap
that would be taken on an unmasked FP exception doesn't take place immediately,
buy the x87 status word is updated.  An "fwait" instruction can be used to
force the trap associated with a pending unmasked x87 exception to be taken,
but that doesn't have much to do with the masked-exception-during-FF-CALL case.

- what the test case (which can be reproduced under Linux) seems to show is
that we're just neglecting to clear the x87 status word when we save its value
in thread-local storage after returning from the call.  Nothing that we do
(use of MMX, etc.) seems to clear the status word, so we report a leftover
FP exception when we finally do something (LOG) that actually checks to see
if an FP exception happened during the most recent FF-CALL.

If we clear the exception bits in the status word (via "fclex") after saving
it in the non SSE2_MATH_LIB case, the problem seems to go away.