[erlang-questions] Strange arithmetic behaviour
Trevor Woollacott [ MTN - Innovation Centre ]
Woolla_T@REDACTED
Thu May 8 11:34:47 CEST 2008
Hi
This occurs in most programming languages due to the way floating point
arithmetic is performed.
If I try it in Python I get:
>>> 123123123123123123123.0 - 123123123123123123121.0
0.0
Take out the decimal place and you get:
>>> 123123123123123123123 - 123123123123123123121
2
Try the following in Erlang:
10.1-0.1-10 = 0.00000e+0
However:
10.1-10-0.1 = -3.60822e-16
It's not a bug, it's a limitation
Regards,
Trevor
________________________________
From: erlang-questions-bounces@REDACTED
[mailto:erlang-questions-bounces@REDACTED] On Behalf Of Gurgen
Tumanian
Sent: Thursday, 08 May 2008 10:42 AM
To: erlang-questions@REDACTED
Subject: [erlang-questions] Strange arithmetic behaviour
Dear all.
There is some kind of strange behaviour that i have noticed when dealing
with big float values.
For example:
123123123123123123123.0 - 123123123123123123121.0 is 0.00000e+0 when i
expect 2.0 or something like that in e notation.
I found this kind of errors with round() and trunc(). for example:
trunc(1233333333333333123333333333.12311111111) is
1233333333333333065680814080.
or
round(1233333333333333123333333333.12311111111) is
1233333333333333065680814080.
furthermore
1233333333333333123333333333.12311111111 =
123333333333333312367575676573.92311111111 matches
I have tested this on R11-B5 and on R12-B1
Is this a big nasty bug, or am i missing something?
Regards
Gurgen Tumanyan
Smart Tech
NOTE: This e-mail message is subject to the MTN Group disclaimer see http://www.mtn.co.za/default.aspx?pid=34411
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20080508/ed8146cb/attachment.htm>
More information about the erlang-questions
mailing list