[erlang-questions] Inter nodes messaging bottleneck

[Eli Cohen]

Hi all,

First, thank you all for your inputs.

I'll try to address all the inputs here:

Regarding the network and the interface:
I've tested the network configuration and the interface between the
machines using iperf tool:
I've reach about 8Gbits bandwidth. I've checked my net.core and
net.ipv4.tcp_X configuration and it seems OK.
I also tried to reduce the window size and run parallel connections - in
the worst case scenario I've reached about 2.5Gbits throughput between the
machines.
To remind you I'm talking about ~120Mbits bottleneck.

Regarding the number of UDP sockets:
The requests to start receiving data dynamic. Each data flow may be
received from several UDP ports.
My way to implement it was dynamically create processes when each may
create several gen_udp's to listen on.
This was my way to implement it. I'll be happy to hear another ideas.
To remind you, when I'm moving the receiving machine - the throughput
between the nodes is doubled. (which is still low, but it proves that the
UDP sockets are not the bottleneck -
at least not the original 120Mbits bottleneck).

Regarding the counter:
This is only a helper for debugging this issue. I've mentioned the counter
to clarify that this is the only activity done in the receiving process
(dst node).
For my understanding storing this counter in internal gen_server State
record is the cheapest way to realize it.

Please advice,
Thanks again

2013/12/10 Jesper Louis Andersen <jesper.louis.andersen@REDACTED
>

>
> On Tue, Dec 10, 2013 at 4:09 PM, Eli Cohen <eli.cohen357@REDACTED> wrote:
>
>> Is there any flags/parameters that may affect this area?
>
>
> First thing: Verify that you can actually get the bandwidth you assume
> between the two nodes in a raw transmit without any Erlang in between.
>
> Secondly, look at the TCP connection between the two machines. On a 10Gig
> interface, it is often the case you need to tune the kernel and the network
> card a bit before you can push data flawlessly.
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20131211/57e6bf84/attachment.htm>