[erlang-questions] High lock contention on dist_tables

Brian Picciano <>
Thu Jun 13 02:28:47 CEST 2013

Thanks for the heads up Lukas! Sorry I stopped responding, we ended up
solving the problem (for now) by drastically cutting down on inter-node
communication in another way, and this thread got lost in my inbox, but I
really appreciate the follow-up!

On Thu, May 16, 2013 at 3:43 AM, Lukas Larsson <> wrote:

> Hello Brian,
> Just letting you know that I have just merged a fix which changes the
> rwlock I mentioned before to an rlock. This should reduce the contention
> which you are seeing if it was caused by many calls to erlang:nodes().
> Lukas
> On Tue, Apr 23, 2013 at 9:01 PM, Scott Lystig Fritchie <
> > wrote:
>> Brian Picciano <> wrote:
>> bp> We have a pool of 3 erlang nodes, all on different servers. Every
>> bp> afternoon, without fail, we start seeing lots of messages between
>> bp> the nodes start having really high latency, on the order of tens of
>> bp> seconds. [...]
>> Brian, it's probably worthwhile to continue chasing the 'lcnt' avenue
>> as you've been corresponding with Lukas...
>> ... but at the same time, I also wonder about "tens of seconds".  My gut
>> says that such delays would require some amazingly high lock contention
>> rates.  Something that can cause such messaging delays much more easily
>> is network congestion/packet loss that triggers TCP slow start.  Many
>> Linux kernels have the RTO_min value at one second, which is the amount
>> of time to wait before entering slow start state.
>> If network packet loss is a problem, this blog posting can explain one
>> reason why it's happening:
>> http://www.snookles.com/slf-blog/2012/01/05/tcp-incast-what-is-it/
>> -Scott
>> _______________________________________________
>> erlang-questions mailing list
>> http://erlang.org/mailman/listinfo/erlang-questions
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20130612/e017a351/attachment.html>

More information about the erlang-questions mailing list