[Openmcl-devel] Error in rational -> float conversion

Waldek Hebisch hebisch at math.uni.wroc.pl
Mon Sep 20 12:07:19 PDT 2021


On Mon, Sep 20, 2021 at 10:28:45AM -0700, Ron Garret wrote:
> Is there actually a reason, when converting rationals to
> single-floats, that the computation is done by converting the
> numerator and denominator to single-floats and dividing,
> rather than converting them to double-floats, dividing, and
> then converting the result to a single-float?

If denominator and numerator are exactly representable in
single precision, then there is good reason: such approach
is obviously gives correctly rounded result while it is not
clear for alternatives.  Once numerator or denominator are
not exactly representable in single precision, then things get
tricky.  Converting to double, dividing and then converting
to single in many cases will give correct rounding, but
I would expect some cases of wrong rounding.  One obvious
way to get correct rounding is long division using
integer arithmetic.

-- 
                              Waldek Hebisch



More information about the Openmcl-devel mailing list