[erlang-questions] Network vs native endianness in binary

[Hynek Vychodil]

Hi,
I'm working on some kind special "database" engine and found weird
performance behavior. When I use Erlang default (network, big) integer
representation I got better performance then if I use machine native
(little) representation. It is consistent either for HiPE and byte
code BEAM. It seems weird for me especially in HiPE case when I
imagine it have to swap bytes twice. I expect either BEAM and HiPE is
using native byte order in internal integer representation. When there
is less difference between native (little) HiPE and BEAM
implementation then Erlang default which seems to me that HiPE calls
some sort of Bif same as in BEAM. May be I understand it totally wrong
but it seems to me that there can be some fix in HiPE which will make
using native endianness faster than  Erlang default. I'm planning
rewrite mine pure Erlang prototype to Nif's where using native
endianness will be simpler but I'm just curious what is behind.

There is simple testing module and result:

-module(bln).

-export([test/0, big/1, native/1]).

test() ->
  Big = generate(1048576, <<>>),
  Native = << <<X:32/native>> || <<X:32>> <=Big >>,
  [timer:tc(?MODULE, big, [Big]),
   timer:tc(?MODULE, native, [Native])].

generate(0, Acc) -> Acc;
generate(N, Acc) -> generate(N-1, <<Acc/binary, (random:uniform(4096)-1):32>>).

big(Bin) when is_binary(Bin) -> big(Bin, 0).

big(<<X:32, Rest/binary>>, S) -> big(Rest, S+X);
big(<<>>, S) -> S.

native(Bin) when is_binary(Bin) -> native(Bin, 0).

native(<<X:32/native, Rest/binary>>, S) -> native(Rest, S+X);
native(<<>>, S) -> S.

17> c(bln, [native, {hipe,[o3]}]).
{ok,bln}
18> bln:test().
[{8322,2146938992},{34700,2146938992}]
19> bln:test().
[{8536,2143318954},{35553,2143318954}]
20> c(bln).
{ok,bln}
21> bln:test().
[{14896,2145208908},{41116,2145208908}]
22> bln:test().
[{14765,2145949712},{41943,2145949712}]

I'm getting similar results for mine much more complicated POC.
(Anyway 126Mr/s either 30Mr/s is impressive.)

Cheers

-- 
--Hynek (Pichi) Vychodil