[erlang-questions] term_to_binary and large data structures
Wed Jun 27 23:33:45 CEST 2018
On 2018-06-27 16:39, zxq9@REDACTED wrote:
> On 2018年6月27日水曜日 16時19分14秒 JST Aaron Seigo wrote:
> If you send huge messages you are killing your heartbeats. That doesn't
> play out very well.
Yes, we've noticed ;)
> Instead, consider opening a separate TCP connection between nodes for
> tranfer of large data. That way heartbeats and small inter-node
> don't interfere with your huge bulk message transfers.
That would be fine, and quite doable ... but it means everyone having
similar issues needs to do the same. Why are we using Erlang, again?
Yes, for ease of use of things like concurrency and distribution. When
those things break, the reason to use Erlang evaporates. It would be
better to address the issues (even if it takes a while and some effort
to do so!) then to work around it.
That said .. the recent work on top of 21 to allow out-of-order sending
of messages, including during chunked message transmission, to avoid
head-of-line blocking is really promising and should, once finalized and
upstreamed, resolve this particular issue.
> (Yes, this is totally the kind of thing that there should be a library
> for, but it isn't something there is a lot of time for unpaid work on.
> That sucks, but there it is.)
IMHO it should be part of the distribution system itself, one way or the
other, as using a library to work *around* distribution (and which
itself would need to follow distribution to be transparent) is a little
There are enough of us working on paid time these days to make it
happen, as long as we know what needs doing and what is acceptable. :)
More information about the erlang-questions