options for high-throughput connections between nodes

[Dave Cottlehuber]

I'm investigating how to maximise pushing mostly loosely ordered erlang terms (via `term_to_binary/1` and friends) between a mesh of erlang nodes.

A single TCP stream isn't sufficient, so I'm expecting using either UDP or multiple TCP streams, to push up throughput. I don't expect the usual erlang distribution protocol to be suitable, because of the risk of congestion and head-of-line blocking of distribution-related traffic.

Assume LFN scenario (Long Fat Pipes), up to 3-10GiB/s bandwith, usual inter-continental latency as default. No numbers yet, just "as much as possible / closest to theoretical maximum bandwidth". 

- zeromq: https://github.com/zeromq/chumak | https://github.com/zeromq/erlzmq2 | https://github.com/lukaszsamson/erlzmq
- osiris: https://github.com/rabbitmq/osiris
- nanomsg: https://github.com/basho/enm
- https://www.erlang.org/doc/man/gen_sctp.html

Has anybody other suggestions to add to the list?

thanks
Dave