[erlang-questions] Strange arithmetic behaviour

[Gleb Peregud]

These are results from Python interactive shell:

>>> 123123123123123123123.0 - 123123123123123123121.0
0.0
>>> print '%f' % math.floor(1233333333333333123333333333.12311111111);
1233333333333333065680814080.000000
>>> print '%f' % 1233333333333333123333333333.12311111111
1233333333333333065680814080.000000

As you can see results are the same. I think it is because of internal
IEEE-something floating point number representation. As you can see in
the last example, number "1233333333333333123333333333.12311111111" is
converted to "1233333333333333065680814080.000000" without applying to
it any operations. Probably it is because internal float
representation does not allow this precision in this particular
situation

On 5/8/08, Gurgen Tumanian <tumanian@REDACTED> wrote:
> Dear all.
>
> There is some kind of strange behaviour that i have noticed when dealing
> with big float values.
> For example:
>
>  123123123123123123123.0 - 123123123123123123121.0 is 0.00000e+0  when i
> expect 2.0 or  something like that in e notation.
> I found this kind of errors with round() and trunc(). for example:
> trunc(1233333333333333123333333333.12311111111) is
> 1233333333333333065680814080.
> or
> round(1233333333333333123333333333.12311111111) is
> 1233333333333333065680814080.
>
> furthermore
> 1233333333333333123333333333.12311111111 =
> 123333333333333312367575676573.92311111111 matches
>
>
> I have tested this on R11-B5 and on R12-B1
>
> Is this a big nasty bug, or am i missing something?
>
> Regards
> Gurgen Tumanyan
> Smart Tech
>


-- 
Gleb Peregud
http://gleber.pl/

Every minute is to be grasped.
Time waits for nobody.
-- Inscription on a Zen Gong