[erlang-questions] Bug? Pretty-printing floating point values
Per Hedeland
per@REDACTED
Wed Nov 21 17:59:19 CET 2007
"Christian S" <chsu79@REDACTED> wrote:
>
>Doesn't look like the number of significant digits when read are stored.
Undoubtedly true, but not necessarily relevant.
>I wouldn't call it a bug. It is how floating point numbers work.
>They're very exact with fractions that are a sum of limited numbers of
>negative powers of two though.
A reasonable question could be why float_to_list/1 generates 20 digits
when a 64-bit float (a.k.a. C double), which is what is used internally,
only can hold 15-16 worth of them - I don't know off-hand what a 128-bit
float would have, but presumably significantly more than 20, so it's not
that either. I guess way back in the dark ages, someone thought that 20
was a nice and even number (I hope it wasn't me:-). The 6.30000 form is
of course just the ~p/~w formatting.
--Per Hedeland
Eshell V5.5.5 (abort with ^G)
1> F=6.30000000000000000000000000000.
6.30000
2> float_to_list(F).
"6.29999999999999982236e+00"
3> io:format("~p~n",[F]).
6.30000
ok
4> io:format("~.16g~n",[F]).
6.300000000000000
ok
5> io:format("~.17g~n",[F]).
6.2999999999999998
ok
More information about the erlang-questions
mailing list