[erlang-questions] Hipe and Binary - Bitstring

[Frank Muller]

Can someone shed some light on the difference between min_heap_size &
*min_bin_vheap_size*
on how to tweak them per process to tune VM’s perfs?

Thanks

Hi John,
>
> *problem solved!*
>
> The secret is: process_flag(min_heap_size,1024*1024*10),process_flag(
> *min_bin_vheap_size*,1024*1024*10*10),
>
> with this i get *without native* 1,000,000 steps:
>
> #{gc_major_end => 8,gc_major_start => 8,gc_max_heap_size => 0,gc_minor_end => 85,gc_minor_start => 85}
>
> Performance is 100 time faster, the missing factor 4 comes from hipe itself!
>
> Very nice!
>
> Oliver
>
> On 28.03.19 06:50, Oliver Bollmann wrote:
>
> Hi John,
>
> gc tracer(native) *1,000,000* steps:
>
> #{gc_major_end => 36,gc_major_start => 36,gc_max_heap_size => 0,gc_minor_end => 116,gc_minor_start => 116}
>
> gc tracer(*NOT* native) *1,000* steps:
>
> #{gc_major_end => 35,gc_major_start => 35,gc_max_heap_size => 0,gc_minor_end => 1202,gc_minor_start => 1202}
>
> Oliver
>
>
>
> On 27.03.19 17:22, John Högberg wrote:
>
> Nevermind, I'm blind, I just noticed "UseProf" now. I may need more coffee
> :o)
>
> It's possible that the native code generates less garbage on the heap,
> causing fewer GCs, which will be a lot faster if your processes have a lot
> of live data as it won't have to copy it over and over. Try comparing how
> many garbage collections the process has gone through with process_info(Pid,
> garbage_collection), maybe it will provide some clue.
>
> /John
>
> On Wed, 2019-03-27 at 16:31 +0100, John Högberg wrote:
>
> Hi Oliver,
>
> Have you tried comparing performance without eprof?
>
> eprof uses tracing to figure out which functions take a long time to run,
> which adds considerable overhead to small functions that are repeated
> extremely often. HiPE doesn't support tracing at all, so that overhead
> simply disappears when the module is native-compiled.
>
> Regards,
> John Högberg
>
> On Wed, 2019-03-27 at 16:18 +0100, Oliver Bollmann wrote:
>
> Hi John,
>
> indeed, on standalone the factor is about 3.7 only :-(
>
> Attached the module i used. The code is part of:
> https://gitlab.com/Project-FiFo/DalmatinerDB/bitmap
>
> I wonder, where comes the boost?
>
> Facts: OS OSX 10.14.3(64GB)
>           Erlang 20.3.18,
>           the "boost" module use a lot of process directory (about 10GB,
> almost of this are binaries!)
>
> Any hints?
>
> Oliver
> On 27.03.19 13:04, John Högberg wrote:
>
> Hi Oliver,
>
> I've tried to reproduce this discrepancy on my machine, but I can only see
> a modest difference on latest OTP 21 (the results are in microseconds):
>
> Erlang/OTP 21 [erts-10.3.1] [source] [64-bit] [smp:24:24] [ds:24:24:10] [async-threads:1] [hipe]
>
> Eshell V10.3.1  (abort with ^G)
>
> 1> c(t, []).
>
> {ok,t}
>
> 2> t:bench(one).
>
> 15957304
>
> 3> t:bench(union).
>
> 559470
>
> 4> c(t, [native]).
>
> {ok,t}
>
> 5> t:bench(one).
>
> 3611371
>
> 6> t:bench(union).
>
> 501871
>
>
> I've attached the source code I used for this test, am I missing something?
>
> Regards,
> John Högberg
>
> On Wed, 2019-03-27 at 10:09 +0100, Oliver Bollmann wrote:
>
> I use, with binaries like <<1:1000000>>,
>
> one(F,<<Size:64/unsigned, Bitmap:Size/bitstring, _/bitstring>>) ->
>
>   one(F,Bitmap,0,[]).
>
> one(F, <<0:1, R/bitstring>>, N, Acc) ->
>
>   one(F, R, N + 1, Acc);
>
> one(F, <<1:1, R/bitstring>>, N, Acc) ->
>
>   one(F, R, N + 1, [F(N) | Acc]);
>
> one(_, <<>>, _, Acc) -> Acc.
>
> union(<<Size:64/unsigned, L:Size/unsigned, P/bitstring>>,
>
>     <<Size:64/unsigned, R:Size/unsigned, _/bitstring>>) ->
>
>   <<Size:64/unsigned, (L bor R):Size/unsigned, P/bitstring>>.
>
> and call this functions 1,000,000 times, this takes for 1,000 calls about
> 20 minutes,
>
> if i compile with native -compile([native,{hipe, o2}]) it takes 3 seconds
> for 1,000 calls, so it is about 400x faster !!
>
> OS: OSX
>
> What is the secret?
>
> --
>
> Grüße
>
> Oliver Bollmann
>
> _______________________________________________
>
> erlang-questions mailing list
>
> erlang-questions@REDACTED
>
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
> _______________________________________________
>
> erlang-questions mailing list
>
> erlang-questions@REDACTED
>
> http://erlang.org/mailman/listinfo/erlang-questions
>
> --
>
> Grüße
>
> Oliver Bollmann
>
> --
> Grüße
> Oliver Bollmann
>
>
> _______________________________________________
> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions
>
> --
> Grüße
> Oliver Bollmann
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20190328/0dc0abf4/attachment.htm>