[erlang-questions] round(1.0e23) ≠ 10²³

[Erik Søe Sørensen]

Hi there,

There are certain common misconceptions around floating-point numbers.
The key thing to keep in mind is that a) they have limited precision, as
they are represented in a fixed number of bits, and b) they are represented
in base 2, so numbers which look nice in base 10 and "feel" as if they
should not suffer from precision loss, may surprise by doing so.

In your case, notice that for 10²³ to be represented accurately takes >3*23
= 69 bits -- and Erlang floats are represented in 64 bits (IEEE 754 double
precision / C's "double") -- which means that even if all 64 bits were used
for the mantissa, the number could not be represented accurately. In fact,
there are only 52+1 bits for the mantissa, so the problem starts already at
around 10^16:
  round(10000000000000001.0) => 10000000000000000.


Hoping this helps,
/Erik


2013/4/6 Pierre Fenoll <pierrefenoll@REDACTED>

> Hi,
>
> I have notice that on both my R16B 64bit machines (Archlinux & OSX),
> round(1.0e23) = 99999999999999991611392
> Instead of 10²³ as it should.
>
> I think it's a matter of float to int.
> Do you guys have any idea? Am I doing something wrong?
>
> Cheers,
> Pierre
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20130406/dbb95c46/attachment.htm>