[erlang-questions] Distribution: small packets

[Danil Zagoskin]

Maybe this problem is related to how the application works.

Each node may host a shard of service.
Application contains a process (shard manager) which registers in global
registry with a key like {shard, 2}.
Each shard contains thousands (millions planned) of processes, registered
locally in gproc as {n, l, {worker, <<"worker_name">>}}
Access to the local worker (say, worker:access/2) is pair of
gproc:lookup_pid and gen_server:call
Access to any worker is determining shard number (crc32(Name) rem Count), N
= node(global:whereis_name({shard, 2})), rpc:call(N, worker, access, [...])

So there are two operations related to distribution — global:whereis_name
and rpc:call.

Does any of them force sending of tcp-push? If yes, how can I change this
behavior by cost of couple of millisecods latency?


On Tue, Jul 8, 2014 at 11:20 PM, Danil Zagoskin <z@REDACTED> wrote:

> Hi!
>
> There is enormous packet-per-second in my distributed erlang setup.
>
> Under heavy load in single distribution socket and in one direction
> tcpdump shows rates like 40..100 packets with tcp-push flag set per
> millisecond.
> Size of majority of packets is 47..112 bytes while MTU on network
> interface is 8950 (jumbo-frames).
>
> If distribution driver aggregated messages to fit MTU it would be 100
> times less packets in network.
> Given cluster of 3 nodes (very small) and both directions we get about
> extra 300K PPS which causes packet drops and tcp retransmits (thus
> increasing latency a lot).
>
> Is it possible to make erlang distribution push packets less often (one
> millisecond would be enough for me)?
>
> I'm sure this is not of net ticks because tracing on dist_util:con_loop
> process shows quite low {_, tick} message rate.
>
> Erlang/OTP version used is 17.0.
>
> --
> Danil Zagoskin | z@REDACTED
>



-- 
Danil Zagoskin | z@REDACTED
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20140709/0c5563e9/attachment.htm>