[erlang-questions] round(1.0e23) ≠ 10²³
Richard A. O'Keefe
Sun Apr 7 23:55:02 CEST 2013
On 7/04/2013, at 8:34 AM, Pierre Fenoll wrote:
> I have notice that on both my R16B 64bit machines (Archlinux & OSX),
> round(1.0e23) = 99999999999999991611392
> Instead of 10²³ as it should.
WHy do you think that?
> I think it's a matter of float to int.
> Do you guys have any idea? Am I doing something wrong?
Yes I do and yes you are.
(10 raisedToInteger: 23) asDouble rounded => 99999999999999991611392
in VisualWorks Smalltalk. There's nothing wrong with round():
1.0e23 *is* 99999999999999991611392.0.
This has nothing to do with R16B, nothing to do with Erlang, and nothing
to do with whether your machine is 32-bit or 64-bit or what operating
system it is running. It's all about the precision of IEEE doubles.
Let's see what's happening.
(10 raisedToInteger: 23) printStringRadix: 16 => '152D02C7E14AF6800000'
99999999999999991611392 printStringRadix: 16 => '152D02C7E14AF6000000'
Yep, 10**23 has *just* too many bits between its most significant 1 and
its least significant 1 to fit into an IEEE double. This number needs
a 54-bit significand, but IEEE doubles offer only 53.
More information about the erlang-questions