[erlang-questions] Any way to correct the round off errors?

[Richard Kelsall]

> When subtracting in Erlang: 0.92915-0.92945 we should get -0.0003 but Erlang
> gives: -2.9999999999996696e-4   (when doing 92915-92945 Erlang gives -30 so
> that's ok).
> 
> Anyway to make make it give -0.0003 ?, and in general make it give more
> accurate answers?

That's an interesting question. The other replies tell us it's a 64 bit
Double floating point representation :

http://en.wikipedia.org/wiki/Double_precision_floating-point_format

which has, according to Wikipedia, "about 16 decimal digits" of
precision from the 52 bits. Their example is correct up to 16 digits :

0.33333333333333331482
                  *
   1234567890123456

but yours is only correct up to 13 digits :

2.9999999999996696e-4
              *
1 234567890123456

so we probably shouldn't trust more than 12 digits of precision because
each calculation will lose some precision from the end of the number.
Different calculations will lose different amounts of precision and
very long sequences of calculations could reduce the number of correct
digits below 12.

You might think it would be nice to have every calculation keep a record
in the double of how many digits we can trust. If we did this the output
routine could give the exact answer to your calculation. But this would
be very difficult to calculate and would consume processor power at
every step. So instead floating point routines rely on the programmer
to know about floating point accuracy and you always have to round the
output appropriately. If you round that answer to 12 digits it is
exactly correct.

In general always use a number representation that can cope with numbers
much bigger and with much more accuracy than you are expecting because
bugs in this area are nasty.


Richard.