[erlang-questions] round(1.0e23) ≠ 10²³

[Richard A. O'Keefe]

On 7/04/2013, at 8:34 AM, Pierre Fenoll wrote:

> Hi,
> 
> I have notice that on both my R16B 64bit machines (Archlinux & OSX),
> round(1.0e23) = 99999999999999991611392
> Instead of 10²³ as it should.

WHy do you think that?
> 
> I think it's a matter of float to int.
> Do you guys have any idea? Am I doing something wrong?

Yes I do and yes you are.

(10 raisedToInteger: 23) asDouble rounded => 99999999999999991611392

in VisualWorks Smalltalk.  There's nothing wrong with round():
1.0e23 *is* 99999999999999991611392.0.
This has nothing to do with R16B, nothing to do with Erlang, and nothing
to do with whether your machine is 32-bit or 64-bit or what operating
system it is running.  It's all about the precision of IEEE doubles.

Let's see what's happening.
(10 raisedToInteger: 23) printStringRadix: 16 => '152D02C7E14AF6800000'
 99999999999999991611392 printStringRadix: 16 => '152D02C7E14AF6000000'

Yep, 10**23 has *just* too many bits between its most significant 1 and
its least significant 1 to fit into an IEEE double.  This number needs
a 54-bit significand, but IEEE doubles offer only 53.