[erlang-questions] How to tune erlang memory allocator to use 25 GB of ram and stop munmaping it?

[Max Lapshin]

Right now I fight with problem of streaming 10 Gbit/s of video and more
from single server.

I have met different problems (mostly about locking into single process
like with file:read_file),
but right now I have one question, it is about memory allocation.

I've started strace against one of threads running scheduler and saw that
it makes lot of mmap/munmap.

For example during second 1376487042 it has called mmap 72 times with
sizes 1052672, 2101248.

Such lines are very interesting for me:

1376487042.167246 mmap(NULL, 1052672, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe8f49bf000
1376487042.174690 munmap(0x7fe90c6ff000, 1052672) = 0


erlang asked system for one megabyte of memory and after 7 milliseconds
returned it back.

To make it clear: erlyvideo is reading 700-800 megabytes per second from
several hard drives, repack them a bit and sends to network via TCP sockets.



It is possible to tell erlang that it can use 20GB or 30 GB or ram and
don't return it back at all?

I suppose that such often allocation/deallocation leads to high cpu usage
of process kswapd0 that (perhaps) tries to do something with such big
amount of freed pages.

I have looked at erts_alloc.html page. Well, it is scary a bit, just like
when I've looked into the airplane cabin. Should I change any options from
this page or I shouldn't touch memory allocator at all?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20130814/82c55d32/attachment.htm>