[erlang-questions] Memory allocation HOWTO from Fred

[Max Lapshin]

https://blog.heroku.com/archives/2013/11/7/logplex-down-the-rabbit-hole

Fred, at first: thank you for this tutorial.

It is not just a blogpost, it looks more like a paper into journal =)



I've experiencing similar problems in erlyvideo, because in some places
reallocating memory on gigabits per second speed creates lots of binaries.

For example: if I read from tcp socket with active,once mode, I get 1400
byte blocks, which are big enough to get into big heap, but too small to
handle them and it ends in reallocation and copying hundreds times per
second.

There is no  {active, 100K} mode that will send messages of 100K bytes, so
the only way out is something like:

handle_info({tcp, Sock, Bin}, State) ->
  {ok, Bin2} = gen_tcp:recv(Sock, 50000),
  handle(<<Bin/binary, Bin2/binary>>)

Without such primitive patch I get about 500 new binaries per second and
CPU is melting out.
I don't know why, but it is clear: when there are 100 processes in system
that creates about 500 small binaries per second, beam.smp is consuming lot
of CPU power.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20131108/1a238183/attachment.htm>