[erlang-questions] Inter-node communication bottleneck

[Jihyun Yu]

Hi,

There is a prior discusson[1] about inter-node communication bottleneck,
and I experienced same issue on inter-node messaging. I dig into the
issue and it found that there is a lock on inter-node messaging[2] which
causes bottleneck on sending messages to single node. With simple
experiment, I found that there is a limitation on number of messages per
second, not bandwidth. 1Gbps link could be easily satuated by messaging
betweeen two erlang nodes if message size is large enough, e.g. 4KB
binary message. However if message is small, e.g. single integer term,
number of messages per seconds is limited about 200~300k/sec.

Is there any effort to solve the problem? In fact the problem could be
solved by using external channel like multiple TCP connection pool for
rpc, but I believe using erlang messaging keeps code much simpler.
Actually for me the problem is quite embrassing, because I thought
scalable messaging is a key feature and strength of Erlang.


[1] http://erlang.org/pipermail/erlang-questions/2013-December/076232.html
[2] https://github.com/erlang/otp/blob/OTP-17.1/erts/emulator/beam/dist.c#L1768-L1826