[erlang-questions] floating point exception when accuracy exceeds limit

Johan Montelius johanmon@REDACTED
Tue Jan 20 11:57:33 CET 2015


Ahh ignore, was thinking about 6.0e-154

   Johan

On Tue, 20 Jan 2015 11:26:38 +0100, Johan Montelius <johanmon@REDACTED>  
wrote:

>
> This might be no news to anyone who is doing a lot of floating point  
> arithmetics but it made me scratch my head.
>
>
> 7> (6.0e150*6.0e150).
> 3.6000000000000004e301
>
> 8> (6.0e154*6.0e154).
> ** exception error: an error occurred when evaluating an arithmetic  
> expression
>       in operator  */2
>          called as 6.0e154 * 6.0e154
>
>
> Hmm, would it not be better to answer with the highest possible accuracy?
>
>    Johan
>
>


-- 
Assoc. Prof. Johan Montelius
Software and Computer Systems
School of Information and Communication Technology
KTH - Royal Institute of Technology



More information about the erlang-questions mailing list