[erlang-questions] floating point exception when accuracy exceeds limit
Johan Montelius
johanmon@REDACTED
Tue Jan 20 11:57:33 CET 2015
Ahh ignore, was thinking about 6.0e-154
Johan
On Tue, 20 Jan 2015 11:26:38 +0100, Johan Montelius <johanmon@REDACTED>
wrote:
>
> This might be no news to anyone who is doing a lot of floating point
> arithmetics but it made me scratch my head.
>
>
> 7> (6.0e150*6.0e150).
> 3.6000000000000004e301
>
> 8> (6.0e154*6.0e154).
> ** exception error: an error occurred when evaluating an arithmetic
> expression
> in operator */2
> called as 6.0e154 * 6.0e154
>
>
> Hmm, would it not be better to answer with the highest possible accuracy?
>
> Johan
>
>
--
Assoc. Prof. Johan Montelius
Software and Computer Systems
School of Information and Communication Technology
KTH - Royal Institute of Technology
More information about the erlang-questions
mailing list