[erlang-questions] High lock contention on dist_tables
Thu Jun 13 02:28:47 CEST 2013
Thanks for the heads up Lukas! Sorry I stopped responding, we ended up
solving the problem (for now) by drastically cutting down on inter-node
communication in another way, and this thread got lost in my inbox, but I
really appreciate the follow-up!
On Thu, May 16, 2013 at 3:43 AM, Lukas Larsson <lukas@REDACTED> wrote:
> Hello Brian,
> Just letting you know that I have just merged a fix which changes the
> rwlock I mentioned before to an rlock. This should reduce the contention
> which you are seeing if it was caused by many calls to erlang:nodes().
> On Tue, Apr 23, 2013 at 9:01 PM, Scott Lystig Fritchie <
> fritchie@REDACTED> wrote:
>> Brian Picciano <mediocregopher@REDACTED> wrote:
>> bp> We have a pool of 3 erlang nodes, all on different servers. Every
>> bp> afternoon, without fail, we start seeing lots of messages between
>> bp> the nodes start having really high latency, on the order of tens of
>> bp> seconds. [...]
>> Brian, it's probably worthwhile to continue chasing the 'lcnt' avenue
>> as you've been corresponding with Lukas...
>> ... but at the same time, I also wonder about "tens of seconds". My gut
>> says that such delays would require some amazingly high lock contention
>> rates. Something that can cause such messaging delays much more easily
>> is network congestion/packet loss that triggers TCP slow start. Many
>> Linux kernels have the RTO_min value at one second, which is the amount
>> of time to wait before entering slow start state.
>> If network packet loss is a problem, this blog posting can explain one
>> reason why it's happening:
>> erlang-questions mailing list
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the erlang-questions