[erlang-questions] Strange arithmetic behaviour

Trevor Woollacott [ MTN - Innovation Centre ] Woolla_T@REDACTED
Thu May 8 11:34:47 CEST 2008


Hi

 

This occurs in most programming languages due to the way floating point
arithmetic is performed. 

 

If I try it in Python I get:

>>> 123123123123123123123.0 - 123123123123123123121.0

0.0

 

Take out the decimal place and you get:

>>> 123123123123123123123 - 123123123123123123121

2

 

 

Try the following in Erlang:

10.1-0.1-10 = 0.00000e+0

 

However:

10.1-10-0.1 = -3.60822e-16

 

It's not a bug, it's a limitation

 

Regards,

Trevor

________________________________

From: erlang-questions-bounces@REDACTED
[mailto:erlang-questions-bounces@REDACTED] On Behalf Of Gurgen
Tumanian
Sent: Thursday, 08 May 2008 10:42 AM
To: erlang-questions@REDACTED
Subject: [erlang-questions] Strange arithmetic behaviour

 

Dear all.

There is some kind of strange behaviour that i have noticed when dealing
with big float values.
For example:

 123123123123123123123.0 - 123123123123123123121.0 is 0.00000e+0  when i
expect 2.0 or  something like that in e notation.
I found this kind of errors with round() and trunc(). for example: 
trunc(1233333333333333123333333333.12311111111) is
1233333333333333065680814080.
or
round(1233333333333333123333333333.12311111111) is
1233333333333333065680814080.

furthermore 
1233333333333333123333333333.12311111111 =
123333333333333312367575676573.92311111111 matches


I have tested this on R11-B5 and on R12-B1

Is this a big nasty bug, or am i missing something?

Regards
Gurgen Tumanyan
Smart Tech




NOTE: This e-mail message is subject to the MTN Group disclaimer see http://www.mtn.co.za/default.aspx?pid=34411 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20080508/ed8146cb/attachment.htm>


More information about the erlang-questions mailing list