[erlang-questions] Any way to correct the round off errors?
Jayson Vantuyl
kagato@REDACTED
Sun Sep 20 13:43:19 CEST 2009
For the record, I just tested the same math in Python and Ruby. Same
result. Java probably would as well. They all represent real numbers
as IEEE 754 floating point numbers.
The error you're seeing is because the fractions are represented as
binary fractions. The only way you're going to get "precise" numbers
is to use fixed-point arithmetic.
Keep in mind that accurate is in the eye of the beholder. While you
want to represent 3/10000 (which would be more accurate with base-10
fixed point arithmetic), if you wanted something more like 1/65536,
floating point would be dead-on accurate, but fixed point base-10
would have error.
Here's a partial discussion that may be of use. I probably should put
implementing a decimal module on my list of things to do, though,
because this is hardly a solved problem (and would be extra useful for
handling money values).
http://www.trapexit.org/forum/viewtopic.php?p=44093
On Sep 20, 2009, at 4:35 AM, G.S. wrote:
> Hello everyone,
>
> When subtracting in Erlang: 0.92915-0.92945 we should get -0.0003
> but Erlang
> gives: -2.9999999999996696e-4 (when doing 92915-92945 Erlang gives
> -30 so
> that's ok).
>
> Anyway to make make it give -0.0003 ?, and in general make it give
> more
> accurate answers?
>
> Regards,
> -Gene
--
Jayson Vantuyl
kagato@REDACTED
More information about the erlang-questions
mailing list