arithmetic anomaly?

chas@REDACTED chas@REDACTED
Wed Aug 11 07:05:55 CEST 2004


with the erlang shell i get this:

(1024*1024-1) / 1024.
1024.00

the equivalent expression on the same platform (NetBSD on Intel) in
the clisp implementation of common lisp gives me this:

(/ (- (* 1024 1024) 1) 1024.0)
1023.999

ocaml:

# float_of_int (1024*1024-1) /. 1024.0;;
- : float = 1023.99902344

python:

(1024*1024-1) / 1024.
1023.9990234375

the following gives a more expected result in erlang, so i wonder
whether it's not a rounding error somewhere:

(1024*1024-6) / 1024.
1023.99




More information about the erlang-questions mailing list