[erlang-questions] Any way to correct the round off errors?
Sun Sep 20 18:08:40 CEST 2009
Witold Baryluk wrote:
>> so we probably shouldn't trust more than 12 digits of precision because
>> each calculation will lose some precision from the end of the number.
> Substraction (and addition) can lose any number of digits you wish.
I would be horrified if I added two doubles
I have no idea what the IEEE standard specifies, but I can't imagine
anybody ever implementing or using a version that gave this answer.
I would expect at least twelve significant digits of precision.
But if I added these two doubles
I wouldn't be surprised to get
which gives the right answer to 12 significant digits, but loses all
of the significant digits in the second number.The same for subtraction.
More information about the erlang-questions