[erlang-questions] How to tune erlang memory allocator to use 25 GB of ram and stop munmaping it?

[Lukas Larsson]

Hello Max,

You have forgot to mention which version of Erlang/OTP you are running.
Things have changed quite a bit in the last releases of Erlang on this
front.

You can get a lot of information about what is actually allocated by
looking at erlang:system_info(allocator) and
erlang:system_info({allocator,Alloc}). And this information can guide you
in you decision of what flags you might want to set.

In your case it sounds like maybe you end up allocating a lot of sbcs
(single block carriers) instead of getting mbcs (multi block carriers). You
can check this by looking at the mbcs and sbcs section in the info you get
from erlang:system_info({allocator,$type_alloc}). If you notice you have a
lot of sbcs you might want to raise the threashold for when a sbc is used,
+MB**sbct 1024.

You might want to look at if the calls section in any allocator has almost
as many to $type_alloc as it does to mseg_alloc. eg:

1> erlang:system_info({allocator,binary_alloc}).
[{instance,0,
           [{versions,"0.9","3.0"},
 -- snip --
           {calls,[{binary_alloc,0,77},
                    {binary_free,0,76},
                    {binary_realloc,0,0},
                    {mseg_alloc,0,3},
                    {mseg_dealloc,0,2},
                    {mseg_realloc,0,0},
                    {sys_alloc,0,0},
                    {sys_free,0,0},
                    {sys_realloc,0,0}]}]}]

We can in the above see that there were 77 calls to binary_alloc and we did
3 mseg_alloc. One mseg_alloc roughly corresponds to a mmap (there is also a
mseg cache, so the actual number of mmaps could be less). By playing with
sbct and maybe mmbcs (this sets a static mbc that is allocated and kept
from though out the lifetime of the beam) you could get better statistics.
Note that IIRC mmbcs only works for mbcs so make sure that it is mbcs
objects which you are allocating before twiddling with it.

Yet another idea could be to play with the mseg cache parameters, MMamcbf,
MMrmcbf and MMmcs. By making the cache keep more carriers you could up the
cache hit rate and thus reduce mmaps.

Lukas



On Wed, Aug 14, 2013 at 3:41 PM, Max Lapshin <max.lapshin@REDACTED> wrote:

> Right now I fight with problem of streaming 10 Gbit/s of video and more
> from single server.
>
> I have met different problems (mostly about locking into single process
> like with file:read_file),
> but right now I have one question, it is about memory allocation.
>
> I've started strace against one of threads running scheduler and saw that
> it makes lot of mmap/munmap.
>
> For example during second 1376487042 it has called mmap 72 times with
> sizes 1052672, 2101248.
>
> Such lines are very interesting for me:
>
> 1376487042.167246 mmap(NULL, 1052672, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe8f49bf000
> 1376487042.174690 munmap(0x7fe90c6ff000, 1052672) = 0
>
>
> erlang asked system for one megabyte of memory and after 7 milliseconds
> returned it back.
>
> To make it clear: erlyvideo is reading 700-800 megabytes per second from
> several hard drives, repack them a bit and sends to network via TCP sockets.
>
>
>
> It is possible to tell erlang that it can use 20GB or 30 GB or ram and
> don't return it back at all?
>
> I suppose that such often allocation/deallocation leads to high cpu usage
> of process kswapd0 that (perhaps) tries to do something with such big
> amount of freed pages.
>
> I have looked at erts_alloc.html page. Well, it is scary a bit, just like
> when I've looked into the airplane cabin. Should I change any options from
> this page or I shouldn't touch memory allocator at all?
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20130814/78f450cd/attachment.htm>