[erlang-questions] Timers for hibernated processes

[Zvi]

The problem DOESN'T happen with timers disabled (via macro) or when
using only single timer.

I suspect, that timers are not implemented in scalable way in the BEAM
or there are any soft/hard limits (like for number of processes/ports/
ets/etc.)

We monitor message queue length of all of our registered processes and
total number of messages waiting in the VM. All queues are either 0 or
low numbers and CPU usage is 0% or near zero.

Thanks for the suggestion  to use eprof - we'll try it.

On Nov 22, 4:07 pm, Ahmed Omar <spawn.th...@REDACTED> wrote:
> Hi Zvi,
> It would be useful if you can use
> eprof<http://www.erlang.org/doc/man/eprof.html>to get some
> measurements about how time is spent, is it really in starting
> the timer or something else.
>
>
>
>
>
>
>
>
>
>
> On Tue, Nov 22, 2011 at 1:02 PM, Zvi <zvi.avra...@REDACTED> wrote:
> > Hi,
>
> > I'll try to add more info to Ori's post (we work together).
>
> > The code of our tests is public:
>
> > 1. The version with timer per process (maximum we can get ~ 370K
> > concurrent connections out of 500K):
>
> >https://github.com/nivertech/cowboy/blob/ori_acceptor_rate_throttling...
>
> > 2. The version with single timer - broadcasting idle message to all
> > per-connection processes (no problems - we can get 500K  concurrent
> > connections out of 500K):
>
> >https://github.com/nivertech/cowboy/blob/ori_acceptor_rate_throttling...
>
> > We also tried variation of [1] but with hibernate - same results.
> > Both demos started using variation of go.sh script:
>
> >https://github.com/nivertech/cowboy/blob/ori_acceptor_rate_throttling...
>
> > The solution with single timer is not good for us, because the
> > broadcast itself may take some time (seconds even using high
> > priority). Also we have application-level timers in almost every
> > gen_server in  our codebase.
>
> > Right now we experimenting with creating our own timer service, which
> > will reuse one timer for groups of processes.
>
> > So the interesting question is, how timers implemented inside BEAM /
> > stdlib and what's effective way to replace them.
> > Do timers implemented using processes?
>
> > Thanks in advance,
> > Zvi
>
> > On Nov 22, 12:58 pm, Ahmed Omar <spawn.th...@REDACTED> wrote:
> > > Hi Ori,
> > > Can you be a bit more specific how it hurts your performance? Is there
> > only
> > > one process responsible for creating timers? What does your app do in the
> > > first place?
>
> > > On Tue, Nov 22, 2011 at 2:32 AM, ori brost <oribr...@REDACTED> wrote:
> > > > I tried hibernation alone and it did not help, only
> > > > hibernationcombined with reducing timer amount did the trick
> > > > On Mon, Nov 21, 2011 at 9:25 PM, Joel Reymont <joe...@REDACTED>
> > wrote:
> > > > > Are you sure that creating these timers is what's hurting
> > performance?
>
> > > > > My bet is on your processes, try hibernating them.
>
> > > > > Let us know if that helps.
>
> > > > > On Nov 21, 2011, at 7:22 PM, ori brost wrote:
>
> > > > >> We have an erlang program which creates many (300000) timers using
> > > > >> erlang:start_timer. We have noticed in tests that creating these
> > > > >> timers hurts our performance. For each such timer we have a process
> > > > >> that gets a message from it once every 25 seconds, except for
> > waiting
> > > > >> for these messages, the processes are mostly idle. We are working on
> > > > >> reducing the number of timers, but we would also like to know
> > whether
> > > > >> these timers may hurt performance less if the processes would
> > > > >> hibernate until they get a message. Furthermore, is there any
> > > > >> documentation explaining why creating many timers can hurt erlang
> > > > >> performance?
>
> > --------------------------------------------------------------------------
> > > > > - for hire: mac osx device driver ninja, kernel extensions and usb
> > > > drivers
>
> > ---------------------+------------+---------------------------------------
> > > > >http://wagerlabs.com|@wagerlabs |
> > > >http://www.linkedin.com/in/joelreymont
>
> > ---------------------+------------+---------------------------------------
>
> > > > _______________________________________________
> > > > erlang-questions mailing list
> > > > erlang-questi...@REDACTED
> > > >http://erlang.org/mailman/listinfo/erlang-questions
>
> > > --
> > > Best Regards,
> > > - Ahmed Omarhttp://nl.linkedin.com/in/adiaa
> > > Follow me on twitter
> > > @spawn_think <http://twitter.com/#!/spawn_think>
>
> > > _______________________________________________
> > > erlang-questions mailing list
> > > erlang-questi...@REDACTED://
> > erlang.org/mailman/listinfo/erlang-questions
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questi...@REDACTED
> >http://erlang.org/mailman/listinfo/erlang-questions
>
> --
> Best Regards,
> - Ahmed Omarhttp://nl.linkedin.com/in/adiaa
> Follow me on twitter
> @spawn_think <http://twitter.com/#!/spawn_think>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questi...@REDACTED://erlang.org/mailman/listinfo/erlang-questions