[erlang-questions] clarify: why bit syntax so slow + benchmark code

[Per Gustafsson]

Mateusz Berezecki wrote:
> Hello,
> 
> I have attached a benchmark at the end of this e-mail.
> It is a complete mystery to me why it performs so poorly.
> 
> Could please anyone share some ideas what is the
> background behind this behavior?
> 
> Example test results;
> 
> 24.1486134 seconds / 100 Mb (extracting 1 byte)
> 12.8946602 seconds / 100 Mb (extracting 2 bytes)
> 6.37089217 seconds / 100 Mb (extracting 4 bytes)
> 3.19605309 seconds / 100 Mb (extracting 8 bytes)
> 1.57278535 seconds / 100 Mb (extracting 16 bytes)
> 
> The code was compiled without +native flag.
> I don't understand why it performs so poorly .
> 
> The C code can do 600mb in 24 seconds while performing IO from disk
> while erlang code needs 24 seconds for 100mb while reading it from memory.
> 
> The million dollar questions is - Why ?
> 
> 
> regards,
> Mateusz Berezecki
> 


It is slow because for each iteration a two sub-binary structure (about 
5 words) is allocated. This will be optimized in the next release of 
Erlang/OTP, but your code is sub optimal anyway, if you want good 
performance you should write:

test1(<<>>) -> done;
test1(<<_A, Rest/binary>>) ->
        test1(Rest).

That is, you should not first pattern match and create a sub-binary and 
then match against that one.

Per

> P.S.
> Benchmark code
> 
> -module (benchmark).
> -export([bitsyntax/1, test1/1, test2/1, test4/1, test8/1, test16/1]).
> 
> test1(<<>>) -> done;
> test1(Bin) when is_binary(Bin) ->
>         <<A:1/binary, Rest/binary>> = Bin,
>         <<_A1:8>> = A,
>         test1(Rest).
> 
> test2(<<>>) -> done;
> test2(Bin) when is_binary(Bin) ->
>         <<A:2/binary, Rest/binary>> = Bin,
>         <<_A1:8, _A2:8>> = A,
>         test2(Rest).
> 
> test4(<<>>) -> done;
> test4(Bin) when is_binary(Bin) ->
>         <<A:4/binary, Rest/binary>> = Bin,
>         <<_A1:8,_A2:8,_A3:8,_A4:8>> = A,
>         test4(Rest).
> 
> test8(<<>>) -> done;
> test8(Bin) when is_binary(Bin) ->
>         <<A:8/binary, Rest/binary>> = Bin,
>         <<_A1:8,_A2:8,_A3:8,_A4:8, _A5:8, _A6:8, _A7:8, _A8:8>> = A,
>         test8(Rest).
> 
> test16(<<>>) -> done;
> test16(Bin) when is_binary(Bin) ->
>         <<A:16/binary, Rest/binary>> = Bin,
>         <<_A1:8,_A2:8,_A3:8,_A4:8, _A5:8, _A6:8, _A7:8, _A8:8, _A9:8, _A10:8,
> _A11:8,_A12:8, _A13:8, _A14:8, _A15:8, _A16:8>> = A,
>         test16(Rest).
> 
> bitsyntax(N) when is_integer(N), N > 0 ->
>         K = lists:seq(1, N * 16),
>         Bin = list_to_binary(lists:map(fun(X) -> <<X:8>> end, K)),
> 
>         io:format("Running tests for N = ~p.~n", [N * 16]),
>         {T1, _} = timer:tc(?MODULE, test1, [Bin]),
>         {T2, _} = timer:tc(?MODULE, test2, [Bin]),
>         {T3, _} = timer:tc(?MODULE, test4, [Bin]),
>         {T4, _} = timer:tc(?MODULE, test8, [Bin]),
>         {T5, _} = timer:tc(?MODULE, test16, [Bin]),
> 
>         io:format("T1 = ~p.~n", [T1]),
>         io:format("T2 = ~p.~n", [T2]),
>         io:format("T3 = ~p.~n", [T3]),
>         io:format("T4 = ~p.~n", [T4]),
>         io:format("T5 = ~p.~n", [T5]).
> 
>