[erlang-questions] Hipe and Binary - Bitstring

[John Högberg]

On Wed, 2019-03-27 at 21:30 +0100, Kostis Sagonas wrote:
> On the other hand, I would not call the performance difference
> between 
> BEAM and HiPE that you observed "modest".  Four times faster
> execution 
> is IMO something that deserves a better adjective.
> 
> Kostis

Yes, it's a very impressive improvement. "Modest" was in relation to
that 400x number and I should've been clearer about that, "reasonable
difference" would have been better wording.

On Thu, 2019-03-28 at 08:34 +0100, Oliver Bollmann wrote:
> Hi John,
> problem solved!
> The secret is:
> process_flag(min_heap_size,1024*1024*10),process_flag(min_bin_vheap_s
> ize,1024*1024*10*10),
> with this i get without native 1,000,000 steps:
> #{gc_major_end => 8,gc_major_start => 8,gc_max_heap_size =>
> 0,gc_minor_end => 85,gc_minor_start => 85}
> 
> Performance is 100 time faster, the missing factor 4 comes from hipe
> itself!
> 
> Very nice!
> 
> Oliver

I'm glad it worked out!

However, you're still going to copy those ~2GB of live data when a full
GC finally happens, and I think you should consider reducing that
figure. Do you really need all that data in one process?

On Thu, 2019-03-28 at 08:55 +0100, Frank Muller wrote:
> Can someone shed some light on the difference between min_heap_size
> & min_bin_vheap_size
> 
> on how to tweak them per process to tune VM’s perfs?
> 
> 
> Thanks

On the process heap, off-heap binaries are essentially just a small
chunk with a pointer and size, so if we decided to GC based on the
process heap alone we would keep an unreachable 1GB binary alive for
just as long as a 1KB one (all else equal), which is a bit suboptimal.

We therefore track the combined size of all our off-heap data and GC
when they exceed the "virtual binary heap size," even if the process
heap nowhere near full. This "virtual binary heap" grows and shrinks
much like the ordinary process heap, and the min_bin_vheap_size option
is analogous to min_heap_size.

In general you shouldn't need to play around with these settings, but
if you have a process that you know will grow really fast then there
may be something to gain by bumping its minimum heap size. I don't
recommend doing this without careful consideration though.

http://erlang.org/doc/efficiency_guide/processes.html#initial-heap-size

/John