[erlang-questions] Hipe and Binary - Bitstring

[Oliver Bollmann]

Hi John,

gc tracer(native) *1,000,000* steps:

#{gc_major_end => 36,gc_major_start => 36,gc_max_heap_size => 0,gc_minor_end => 116,gc_minor_start => 116}

gc tracer(*NOT*  native)*1,000*  steps:

#{gc_major_end => 35,gc_major_start => 35,gc_max_heap_size => 0,gc_minor_end => 1202,gc_minor_start => 1202}

Oliver



On 27.03.19 17:22, John Högberg wrote:
> Nevermind, I'm blind, I just noticed "UseProf" now. I may need more 
> coffee :o)
>
> It's possible that the native code generates less garbage on the heap, 
> causing fewer GCs, which will be a lot faster if your processes have a 
> lot of live data as it won't have to copy it over and over. Try 
> comparing how many garbage collections the process has gone through 
> with process_info(Pid, garbage_collection), maybe it will provide some 
> clue.
>
> /John
>
> On Wed, 2019-03-27 at 16:31 +0100, John Högberg wrote:
>> Hi Oliver,
>>
>> Have you tried comparing performance without eprof?
>>
>> eprof uses tracing to figure out which functions take a long time to 
>> run, which adds considerable overhead to small functions that are 
>> repeated extremely often. HiPE doesn't support tracing at all, so 
>> that overhead simply disappears when the module is native-compiled.
>>
>> Regards,
>> John Högberg
>>
>> On Wed, 2019-03-27 at 16:18 +0100, Oliver Bollmann wrote:
>>>
>>> Hi John,
>>>
>>> indeed, on standalone the factor is about 3.7 only :-(
>>>
>>> Attached the module i used. The code is part of: 
>>> https://gitlab.com/Project-FiFo/DalmatinerDB/bitmap
>>>
>>> I wonder, where comes the boost?
>>>
>>> Facts: OS OSX 10.14.3(64GB)
>>>           Erlang 20.3.18,
>>>           the "boost" module use a lot of process directory (about 
>>> 10GB, almost of this are binaries!)
>>>
>>> Any hints?
>>>
>>> Oliver
>>>
>>> On 27.03.19 13:04, John Högberg wrote:
>>>> Hi Oliver,
>>>>
>>>> I've tried to reproduce this discrepancy on my machine, but I can 
>>>> only see a modest difference on latest OTP 21 (the results are in 
>>>> microseconds):
>>>>
>>>> Erlang/OTP 21 [erts-10.3.1] [source] [64-bit] [smp:24:24] [ds:24:24:10] [async-threads:1] [hipe]
>>>> Eshell V10.3.1  (abort with ^G)
>>>> 1> c(t, []).
>>>> {ok,t}
>>>> 2> t:bench(one).
>>>> 15957304
>>>> 3> t:bench(union).
>>>> 559470
>>>> 4> c(t, [native]).
>>>> {ok,t}
>>>> 5> t:bench(one).
>>>> 3611371
>>>> 6> t:bench(union).
>>>> 501871
>>>>
>>>> I've attached the source code I used for this test, am I missing 
>>>> something?
>>>>
>>>> Regards,
>>>> John Högberg
>>>>
>>>> On Wed, 2019-03-27 at 10:09 +0100, Oliver Bollmann wrote:
>>>>>
>>>>> I use, with binaries like <<1:1000000>>,
>>>>>
>>>>> one(F,<<Size:64/unsigned,Bitmap:Size/bitstring,_/bitstring>>) ->
>>>>>    one(F,Bitmap,0,[]).
>>>>> one(F, <<0:1,R/bitstring>>,N,Acc) ->
>>>>>    one(F,R,N +1,Acc);
>>>>> one(F, <<1:1,R/bitstring>>,N,Acc) ->
>>>>>    one(F,R,N +1, [F(N) |Acc]);
>>>>> one(_, <<>>,_,Acc) ->Acc.
>>>>> union(<<Size:64/unsigned,L:Size/unsigned,P/bitstring>>,
>>>>>      <<Size:64/unsigned,R:Size/unsigned,_/bitstring>>) ->
>>>>>    <<Size:64/unsigned, (L bor R):Size/unsigned,P/bitstring>>.
>>>>>
>>>>> and call this functions 1,000,000 times, this takes for 1,000 
>>>>> calls about 20 minutes,
>>>>>
>>>>> if i compile with native -compile([native,{hipe, o2}])it takes 3 
>>>>> seconds for 1,000 calls, so it is about 400x faster !!
>>>>>
>>>>> OS: OSX
>>>>>
>>>>> What is the secret?
>>>>>
>>>>> -- 
>>>>> Grüße
>>>>> Oliver Bollmann
>>>>> _______________________________________________
>>>>> erlang-questions mailing list
>>>>> erlang-questions@REDACTED  <mailto:erlang-questions@REDACTED>
>>>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>>
>>>> _______________________________________________
>>>> erlang-questions mailing list
>>>> erlang-questions@REDACTED
>>>> http://erlang.org/mailman/listinfo/erlang-questions
>>> -- 
>>> Grüße
>>> Oliver Bollmann

-- 
Grüße
Oliver Bollmann

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20190328/e5688300/attachment.htm>