[erlang-questions] High latency when exchanging small messages between different Erlang nodes

[Tristan Sloughter]

What is the frequency of initial messages sent? Are the worker processes mailboxes increasing over time as well?

Tristan

On Thu, Apr 11, 2019, at 13:07, Konstantinos Kallas wrote:
> Hello,

> I have an Erlang application where latency is crucial and a lot of small messages (tuples with an atom and integer) are exchanged between processes in different nodes. 

> The main procedure is that a main process sends a small message to 4 worker processes in other Erlang nodes, the worker processes do some negligible processing, and then they reply back to the main node with a small message. 

> Each separate Erlang node is on a different docker container (generated from the erlang:21 docker image), and all the containers are connected using a standard docker bridge network.

> I have noticed that latency (the time from when the first message is sent, and its replies arrive) linearly increases with time. It starts at 1 second and after 30 seconds of execution latency has become 10 seconds.

> I have tried running all processes on the same erlang node, and then latency is (as expected) a couple milliseconds, so my assumption is that the problem could be caused by one (or more) of the following:

> - Some misconfiguration of the Erlang nodes

> - Some misconfiguration of the docker network/containers

> - Some penalty imposed by the operating system/docker because a lot of small messages are exchanged

> Has anyone encountered this issue, or does anyone know how to configure Erlang nodes (and the operating system) to reduce message latency? 

> Thanks in advance.

> Best,

> Konstantinos

> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20190411/4dc337d2/attachment.htm>