[Openmcl-devel] Error in rational -> float conversion

[Raymond Wiker]

For IEEE-754 single-precision floating-point, the mantissa is held in 23 bits. log10 of 2^23 is about 6.9237, so you should expect less than 7 decimal digits of accuracy. 411071.0 is 7 digits, so anything after that can (and should) be ignored. The difference between 411071.0 and 411071.03 expressed as a float32 is a single bit: the least significant bit of the mantissa.

The reason that coerce gives a slightly worse result is that you compute the value as float32, then convert to float64, and when you print the number, more digits will be printed.


> On 19 Sep 2021, at 15:05, Steven Nunez <steve_nunez at yahoo.com <mailto:steve_nunez at yahoo.com>> wrote:
> 
> Hi folks,
> 
> I just posted an issue regarding what looks like a bug in rational->float conversion <https://github.com/Clozure/ccl/issues/390>, and I'm hoping someone here can try this on 1.12 (preferably on MS Windows), or perhaps provide some insight as to what might be going on here for a quick fix.
> 
> CL-USER> (lisp-implementation-version)
> "Version 1.11.8 (v1.11.8-2-gd411e378) WindowsX8664"
> CL-USER> (float 41107100000541273/100000000000)
> 411071.03
> 
> Hmm. If you work out the decimal points, the float is: 411071.00000541273, that's a long way from .03. Maybe coerce? Nope, that's even worse:
> 
> CL-USER> (coerce 41107100000541273/100000000000 'double-float)
> 411071.03125D0
> 
> Any ideas? That's not a small rounding error.
> 
> Regards,
>     Steve
> _______________________________________________
> Openmcl-devel mailing list
> Openmcl-devel at clozure.com <mailto:Openmcl-devel at clozure.com>
> https://lists.clozure.com/mailman/listinfo/openmcl-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clozure.com/pipermail/openmcl-devel/attachments/20210920/eb7f26e6/attachment.htm>