<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Hi Gleb,<div class=""><br class=""></div><div class="">Hope you don’t mind me digging up this thread, but I was wondering if you could ask you a few questions based on your experience as we are looking at a similar situation where we have 500k (or more) processes active on a single node. Each process kicks off a couple of long running timers using erlang:send_after with dest being self(). We haven’t hit any slowdowns yet, but I am curious to understand your experience better. </div><div class=""><br class=""></div><div class="">Were you using erlang:send_after or the timer module?</div><div class=""><br class=""></div><div class="">How did you eventually resolve your problem with the locking? Did you have to redesign how you were using the 3 to 5 timers per process so the total number of timers was reduced, or did you do something else, such as stagger the timers so they were not all firing at the same time?</div><div class=""><br class=""></div><div class="">Thanks in advance!</div><div class=""><br class=""></div><div class="">Cheers,</div><div class="">Bryan</div><div class=""><br class=""><div apple-content-edited="true" class="">
<div style="orphans: auto; text-align: start; text-indent: 0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="orphans: 2; text-align: -webkit-auto; text-indent: 0px; widows: 2; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">----</div><div style="color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">Bryan Hughes</div><div style="color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span class="Apple-converted-space"> </span><b class="">Go Factory</b></div><div style="color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><a href="http://www.go-factory.net" class="">http://www.go-factory.net</a></div><div style="color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div></div></div></div><div><blockquote type="cite" class=""><div class="">On Sep 22, 2012, at 1:06 AM, Gleb Peregud <<a href="mailto:gleber.p@gmail.com" class="">gleber.p@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><p class="">In my experience, using excessive timers once bit me hard due to lock contention on global timers lock in Erts. Roughly it was about 3-5 timers created/expired for each of around 200k-500k processes at roughly the same moment (a big broadcast to all processes). Fortunately lcnt showed the problem which was slowing down the broadcast tenfold - lock conflicts was at 95% level on timers lock.</p>
<div class="gmail_quote">On Sep 22, 2012 8:07 AM, "Rapsey" <<a href="mailto:rapsey@gmail.com" class="">rapsey@gmail.com</a>> wrote:<br type="attribution" class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class=""><br class=""></div><div class="">3. Is spawning a lot of send_after's or timers a dangerous idea? In case of using timer module, we'll just lose some started timers due to port restart. But something tells me that crash of erlang internal timers is a somewhat more dangerous thing that might even crash the VM.</div>
<div class=""><br class=""></div></blockquote><div class="">How much is a lot of send_afters? My server runs a lot of timers, but very rarely in more than one per process. But there can easily be thousands of timers running in different processes.</div>
I would advise against using timer module. Using the timer module in a few places turned out to be a big mistake. It sometimes just stops working completely. It may be a mistake on my part, but erlang:send_after generally does everything you need and is completely reliable.<div class="">
<br class=""></div><div class=""><br class=""></div><div class="">Sergej </div></div>
<br class="">_______________________________________________<br class="">
erlang-questions mailing list<br class="">
<a href="mailto:erlang-questions@erlang.org" class="">erlang-questions@erlang.org</a><br class="">
<a href="http://erlang.org/mailman/listinfo/erlang-questions" target="_blank" class="">http://erlang.org/mailman/listinfo/erlang-questions</a><br class="">
<br class=""></blockquote></div>
_______________________________________________<br class="">erlang-questions mailing list<br class=""><a href="mailto:erlang-questions@erlang.org" class="">erlang-questions@erlang.org</a><br class="">http://erlang.org/mailman/listinfo/erlang-questions<br class=""></div></blockquote></div><br class=""></div></body></html>