arithmetic anomaly?

[Peter L]

You forgot one line:

1> (1024*1024-1)/1024.
1024.00
2> io:format("~.5f",[v(-1)]).
1023.99902ok

It is just an rounding to 2 decimal thing. The erlang shell
by default gives you 2 decimals and that is correctly rounded
up to 1024.00.

/Peter

>
> with the erlang shell i get this:
>
> (1024*1024-1) / 1024.
> 1024.00
>
> the equivalent expression on the same platform (NetBSD on Intel) in
> the clisp implementation of common lisp gives me this:
>
> (/ (- (* 1024 1024) 1) 1024.0)
> 1023.999
>
> ocaml:
>
> # float_of_int (1024*1024-1) /. 1024.0;;
> - : float = 1023.99902344
>
> python:
>
> (1024*1024-1) / 1024.
> 1023.9990234375
>
> the following gives a more expected result in erlang, so i wonder
> whether it's not a rounding error somewhere:
>
> (1024*1024-6) / 1024.
> 1023.99
>


-- 
Peter Lund
http://lundata.se,+46 70 543 9416