<html><head></head><body><div class="ydp8e652af7yahoo-style-wrap" style="font-family:Helvetica Neue, Helvetica, Arial, sans-serif;font-size:13px;"><div></div>
<div dir="ltr" data-setdir="false">Table parameters are o<span><span style="color: rgb(0, 0, 0); font-family: Helvetica Neue, Helvetica, Arial, sans-serif;">rdered_set, </span></span>concurrent read and write.</div><div><br></div>
</div><div id="yahoo_quoted_9642152805" class="yahoo_quoted">
<div style="font-family:'Helvetica Neue', Helvetica, Arial, sans-serif;font-size:13px;color:#26282a;">
<div>
On Friday, January 17, 2020, 01:10:17 p.m. EST, Led <ledest@gmail.com> wrote:
</div>
<div><br></div>
<div><br></div>
<div><div id="yiv7648156828"><div><div dir="ltr"><div class="yiv7648156828yqt1461054312" id="yiv7648156828yqt21521"><div class="yiv7648156828gmail_quote"><div class="yiv7648156828gmail_attr" dir="ltr"></div><blockquote class="yiv7648156828gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex;"><div><div style="font-family:Helvetica Neue, Helvetica, Arial, sans-serif;font-size:13px;"><div dir="ltr">I am having some performance trouble in a system that does a few queries on a small ets table of around 10,000 records.<br clear="none"><br clear="none">Basically with around 500 concurrent processes, everything is fine, 1500 I start to notice some small degradation, at around 3000 concurrent processes the schedulers grind to a halt, TOP system CPU usage is around 50%, but Erlang scheduler usage (scheduler:<span>utilization</span>) is 100% and capped out on all 40 threads.<br clear="none"><br clear="none">I am guessing the schedulers are all waiting on locks on the ets table. I thought match_object and ets was quite optimized these days, using R22, I am wondering if there is some synchronization/locking issues that could be addressed. Because I mean at 3000 processes maybe hitting that table 10 times per second on average, does not seem like much. 30k match_objects per second, with ongoing inserts. <br clear="none"><br clear="none">Also would there be a way to debug/pinpoint this is the exact issue? I just did A/B testing where I turned off parts of the system, when I turned off the part that does the match_objects on the ETS table, the system ran fine and never deadlocked at 100% scheduler usage. Its also hard to profile, as the system is so locked up the profiler barely runs.<br clear="none"><br clear="none">For now it seems the solution is to rework the architecture and put a second cached view ETS table, so the match_objects can be replaced with key lookups. Which gets filled by a single process running that pulls via match_object from the main table and fills the cache.</div></div></div></blockquote></div></div><div><br clear="none"></div><div></div><span class="yiv7648156828gmail-tlid-translation yiv7648156828gmail-translation" lang="en"><span class="yiv7648156828gmail-" title="">You didn't specify parameters of your table.</span></span><br clear="none"><div><br clear="none"></div><div>-- </div><div class="yiv7648156828gmail_signature" dir="ltr">Led.</div></div></div></div></div>
</div>
</div></body></html>