[erlang-questions] How to tune erlang memory allocator to use 25 GB of ram and stop munmaping it?

[Peer Stritzinger]

Hi,

I'm also interested in reducing the OS allocation and free's but for a 
different use case:

In small embedded systems with no virtual memory (running on RTEMS 
e.g.) there are several problems with malloc/free:

1. its very slow.

2. the system heap fragments.  There is no VM to help.

My favorite behaviour would be for Erlang to either grab all memory 
available or at least a fixed amount on startup and hold on to it 
forever.  Hopefully without fragmenting it too much (I know in case of 
binary heaps that this isn't possible which is a bit disturbing to me).

Can this be achieved by the current ERTS with some parameter settings?

Which direction should I look if I want to add behaviour like this?

BTW:  next best thing is to malloc/free allways same size tiles this 
would avoid outer fragmentation and could be sped up relatively easily.

Cheers,
-- Peer


On 2013-08-15 08:05:01 +0000, Lukas Larsson said:

> Hello,
> 
> Did increasing the max mseg_alloc sbc limit help?
> 
> After thinking a bit more on this issue, I think that my original 
> hypothesis was correct. You are allocating way more sbc than you want 
> to. Moving more of you allocations into mbcs should drastically 
> decrease the number of mmaps needed and also instead of placing one 
> binary within one mmap area, it will place many binaries within a 
> larger mmaped area.
> 
> Do you have any idea of the average sizes of the binaries that you are 
> allocating? If you do I would try to set the sbct to be high enough 
> that maybe 80% of all binaries fit below it. At the same time you 
> should probably also adjust the smbcs and lmbcs to a higher value, 
> maybe something like smbcs = 0.5*sbct lmbcs = 10*sbct. If you know that 
> you will be using a lot of binaries you could also set smbcs to be 
> higher to start with, or decrease the growth stages (mbcgs) so that you 
> faster get to the lmbcs. You will have to play around with sbct, smbcs, 
> lmbcs and mbcgs and measure performance and memory usage to see what is 
> the best fit for you.
> 
> When doing this you can remove the mmsbc config again as you will 
> (hopefully) be using very few sbcs.
> 
> 
> Lukas
> 
> 
> 
> On Wed, Aug 14, 2013 at 4:57 PM, Max Lapshin <max.lapshin@REDACTED> wrote:
> Oh, thank you, will try it.
> 
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions