[erlang-bugs] : Math library defect?

[Raimo Niskanen]

On Sun, May 18, 2008 at 11:17:52PM -0700, Matthew Dempsky wrote:
> On Sun, May 18, 2008 at 10:19 PM, Florian Weimer <fw@REDACTED> wrote:
> > 144115188075855870.0 and 144115188075855872.0 are the same number as far
> > as doubles are concerned.  So if there's a bug, it's in the conversion
> > to decimal, not in pow.
> 
> Hm, good catch.

I can not reproduce the original problem.
Using R12B-2:

1> io:format("~f~n", [math:pow(2,56)]).
72057594037927936.000000
ok
2> io:format("~f~n", [math:pow(2,57)]).
144115188075855872.000000
ok

This is the correct number, but:

3> io:format("~p~n", [math:pow(2,57)]).
144115188075855870.0
ok

...gives the "incorrect" number the original poster claimed.
And it is hard to call that number incorrect since:

4> math:pow(2,57) =:= float(144115188075855870).
true
5> math:pow(2,57) =:= float(144115188075855872).
true

...both are the same floating point number.

6> round(math:pow(2,57)).
144115188075855872

...gives you the right integer.



To elaborate
------------

io:format's ~e, ~f and ~g uses the C library sprintf(, "%.20e, )
for raw conversion and then does string manipulation on the result.

io:format's ~p and ~P prints the minimal number of decimal digits
required to reproduce the correct floating point number, so
it fills with decimal zeros in the low end when required
for large numbers.


> _______________________________________________
> erlang-bugs mailing list
> erlang-bugs@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-bugs

-- 

/ Raimo Niskanen, Erlang/OTP, Ericsson AB