[erlang-bugs] imprecise computation result

[Eric Pailleau]

hello again,
I forgot :
if you needs a 3digits precision at display,
simply * 1000 the computation, and then before displaying result, divide by 1000.
> 2.845 * 2800.
7966.000000000001
but
> 1000 * 2.845 * 2800 / 1000.
7966.0

regards

« Envoyé depuis mon mobile » Eric

Eric Pailleau <eric.pailleau@REDACTED> a écrit :

>hello,
>> 28.0e-1 * 2800.
>7839.999999999999
>but
>> 28 * 2800  / 10.
>7840.0
>
>always tranlate reals into integers and then finally apply division of powers of ten.
>reals are coded on a fix length of bits while decimals of the reals can be infinite.
>it is due to computer architectures and all languages, even C, are impacted.
>regards
>
>« Envoyé depuis mon mobile » Eric
>
>Xu Yifeng <zfsgeek@REDACTED> a écrit :
>
>>Erlang's matchematic is crazy:
>>
>>Erlang R15B03 (erts-5.9.3.1) [source] [smp:2:2] [async-threads:0] [hipe] 
>>[kernel-poll:false]
>>
>>Eshell V5.9.3.1  (abort with ^G)
>>1> 2.8 * 2800.
>>7839.999999999999
>>2>
>>
>>
>>but the result should be 7840.
>>
>>--
>>Xu Yifeng
>>
>>
>>_______________________________________________
>>erlang-bugs mailing list
>>erlang-bugs@REDACTED
>>http://erlang.org/mailman/listinfo/erlang-bugs
>_______________________________________________
>erlang-bugs mailing list
>erlang-bugs@REDACTED
>http://erlang.org/mailman/listinfo/erlang-bugs