[erlang-questions] IEEE-754 subnormals parsing and handling problems and bugs

[Witold Baryluk]

Hello,

I found problem when parsing small numbers

1> list_to_float("0."++lists:duplicate(322, $0)++"1").
1.0e-323
2> list_to_float("0."++lists:duplicate(323, $0)++"1").
0.0

This is contrasting difference to the handling of big numbers

3> list_to_float("1"++lists:duplicate(308, $0)++".0").
1.0e308
4> list_to_float("1"++lists:duplicate(309, $0)++".0").
** exception error: bad argument
     in function  list_to_float/1
        called as list_to_float("1000000...[lots of zeros removed]......000000000.0")

Example 2, shourly should throw error. But actually example 1 also,
becaus it create so called subnormal numbers (aka denormal numbers,
underflow value). Take look at this two examples:

5> list_to_float("0."++lists:duplicate(322, $0)++"123456789").
1.0e-323
6> list_to_float("0."++lists:duplicate(300, $0)++"123456789").
1.23456789e-301


One can check how arithmetic exception handling is working here:

7> 1.0e200 * 1.0e200.
** exception error: bad argument in an arithmetic expression
     in operator  */2
        called as 1.0e200 * 1.0e200

but this doesn't happen for small numbers

8> 0.123456789e-300.
1.23456789e-301
9> 0.123456789e-400.
0.0
10> 0.123456789e-320.
1.235e-321

11> 0.123456789e-100 * 0.123456789e-100.
1.524157875019052e-202
12> 0.123456789e-200 * 0.123456789e-200.
0.0

Why infinities are trapped, but subnormals not? Because there is no good
syntax for NaNs and Infinites, but there is for subnormals and zero?
As of speed in fact subnormal processing is much more slower, than infinities.
This means, that for example adding lots of numbers near 1.0e-320
can be few times slower, than normal numbers. Other oprations, like
square root, trigonometry or multiplications can have even bigger
performance impact.

13> timer:tc(lists, sum, [lists:duplicate(1000000, 0.1e-200)]).
{164756,1.0000000000056682e-195}
14> timer:tc(lists, sum, [lists:duplicate(1000000, 0.1e-300)]).
{163616,9.99999999972789e-296}
15> timer:tc(lists, sum, [lists:duplicate(1000000, 0.1e-310)]).
{238753,1.0000000000158536e-305}
16> timer:tc(lists, sum, [lists:duplicate(1000000, 0.1e-320)]).
{471354,9.98012605e-316}
17> timer:tc(lists, sum, [lists:duplicate(1000000, 0.1e-321)]).
{471283,9.881313e-317}
18> timer:tc(lists, sum, [lists:duplicate(1000000, 0.1e-322)]).
{471398,9.881313e-318}
19> timer:tc(lists, sum, [lists:duplicate(1000000, 0.1e-323)]).
{164053,0.0}

% on my machine (Athlon_. I tested on Intel Core2 in 32-bits, and
% differences are much bigger, up to 20 times slower!


Similar behaviour I found in erl_scan:string/1.

This is also very different behaviour than erl_scan:string/1, used by compiler
to parse shell input and source files.

20> erl_scan:string("1"++lists:duplicate(309, $0)++".0").
{error,{1,erl_scan,{illegal,float}},1}
21> erl_scan:string("1"++lists:duplicate(308, $0)++".0").
{ok,[{float,1,1.0e308}],1}

Everything fine, infinities are trapped.

Unfortunetly, for underflows compiler similary behaves wrong

22> erl_scan:string("0."++lists:duplicate(322, $0)++"1").
{ok,[{float,1,1.0e-323}],1} % should return error
23> erl_scan:string("0."++lists:duplicate(323, $0)++"1").
{ok,[{float,1,0.0}],1} % should return error

Also string:to_float behaves in same way:

24> string:to_float("0."++lists:duplicate(322, $0)++"1").
{1.0e-323,[]} % should return error
25> string:to_float("0."++lists:duplicate(323, $0)++"1").
{0.0,[]} % should return error

26> string:to_float("1"++lists:duplicate(308, $0)++".0").
{1.0e308,[]}
27> string:to_float("1"++lists:duplicate(309, $0)++".0").
{error,no_float}



I think it should be fixed, so more reliable software can be written,
like statistics software (subnormals can for example easilly appear when
summing and multipling small numbers, however it is normally rear, and
should throw error, to not produce bad results, because even using smart
summation algorithms, like Kahan scheme, will not fix this problem).

For sure such floats should not be allowed in source code, or when
parsing from string. And probably also not be allowed to appear using
any builting arithmetic functions.


Another example:

30> math:exp(100).
2.6881171418161356e43
31> math:exp(1000).
** exception error: bad argument in an arithmetic expression
     in function  math:exp/1
        called as math:exp(1000)

infinity trapped, but subnormal not:

32> math:exp(-100).
3.720075976020836e-44
33> math:exp(-1000).
0.0


I also found that scientific notation behaves in same way:

34> list_to_float("1.123456789e-320").
1.1235e-320
35> list_to_float("1.123456789e-330").
0.0


My last argument will be about hardware support. For example many new
ARM processors, supports hardware floating point computations, but often
without support for subnormals! This makes them not fully IEEE-754
compilant, unless running in software floating point mode, which is
slower, especially if it also need to handle subnormals!


I'm using 32-bit cpu:

model name	: AMD Athlon(tm) 
stepping	: 2
cpu MHz		: 1154.450
cache size	: 256 KB
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 1
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx fxsr sse syscall mp mmxext 3dnowext 3dnow up


Erlang version 1:14.b.4-dfsg-1 installed from Debian/GNU Linux testing.
Do not know exact complation flags, but here is small info

1> erlang:system_info(system_version).
"Erlang R14B04 (erts-5.8.5) [source] [rq:1] [async-threads:0] [hipe] [kernel-poll:false]\n"
2> erlang:system_info(system_architecture).
"i486-pc-linux-gnu"
3> erlang:system_info(build_type).
opt
4> erlang:system_info(c_compiler_used).
{gnuc,{4,6,1}}
5> erlang:system_info(debug_compiled).
false
6> erlang:system_info(smp_support).
false
7> erlang:system_info(threads).
true




speed differences of subnormals on Intel Core2 (32-bit mode), same compiler, same options.

1> timer:tc(lists, sum, [lists:duplicate(10000000, 0.1e-300)]).
{278703,9.999999998591641e-295}
3> timer:tc(lists, sum, [lists:duplicate(10000000, 0.1e-310)]).
{1049095,9.999999997470606e-305}
5> timer:tc(lists, sum, [lists:duplicate(10000000, 0.1e-320)]).
{3501330,9.980126046e-315}

% about 12 times slower, and obviously loss of precission.

Beyond infinities, nan, and denormalized numbers, there also
signed zero, but this is handled without problem:

10> list_to_float("0.0") =:= list_to_float("-0.0").
true
11> math:sqrt(list_to_float("-0.0")).
0.0

This is acceptable, because signed zero is mostly usefull with NaN and
infinites support, and because we do not have them, it is good solution
to ignore signed zero problem.


I found no discussion on erlang-questions list in the past, so hope it
is worth discussing.

Also Erlang Reference Manual User's Guide, doesn't mention anything on the matter.

Regards,
Witek




-- 
Witold Baryluk