[erlang-questions] Timers for hibernated processes

Zvi zvi.avraham@REDACTED
Tue Nov 22 13:02:21 CET 2011


Hi,

I'll try to add more info to Ori's post (we work together).

The code of our tests is public:

1. The version with timer per process (maximum we can get ~ 370K
concurrent connections out of 500K):
https://github.com/nivertech/cowboy/blob/ori_acceptor_rate_throttling_141111/test/simple_ws_server.erl

2. The version with single timer - broadcasting idle message to all
per-connection processes (no problems - we can get 500K  concurrent
connections out of 500K):
https://github.com/nivertech/cowboy/blob/ori_acceptor_rate_throttling_141111/test/simple_ws_server2.erl

We also tried variation of [1] but with hibernate - same results.
Both demos started using variation of go.sh script:
https://github.com/nivertech/cowboy/blob/ori_acceptor_rate_throttling_141111/test/go.sh

The solution with single timer is not good for us, because the
broadcast itself may take some time (seconds even using high
priority). Also we have application-level timers in almost every
gen_server in  our codebase.

Right now we experimenting with creating our own timer service, which
will reuse one timer for groups of processes.

So the interesting question is, how timers implemented inside BEAM /
stdlib and what's effective way to replace them.
Do timers implemented using processes?

Thanks in advance,
Zvi

On Nov 22, 12:58 pm, Ahmed Omar <spawn.th...@REDACTED> wrote:
> Hi Ori,
> Can you be a bit more specific how it hurts your performance? Is there only
> one process responsible for creating timers? What does your app do in the
> first place?
>
>
>
>
>
>
>
>
>
> On Tue, Nov 22, 2011 at 2:32 AM, ori brost <oribr...@REDACTED> wrote:
> > I tried hibernation alone and it did not help, only
> > hibernationcombined with reducing timer amount did the trick
> > On Mon, Nov 21, 2011 at 9:25 PM, Joel Reymont <joe...@REDACTED> wrote:
> > > Are you sure that creating these timers is what's hurting performance?
>
> > > My bet is on your processes, try hibernating them.
>
> > > Let us know if that helps.
>
> > > On Nov 21, 2011, at 7:22 PM, ori brost wrote:
>
> > >> We have an erlang program which creates many (300000) timers using
> > >> erlang:start_timer. We have noticed in tests that creating these
> > >> timers hurts our performance. For each such timer we have a process
> > >> that gets a message from it once every 25 seconds, except for waiting
> > >> for these messages, the processes are mostly idle. We are working on
> > >> reducing the number of timers, but we would also like to know whether
> > >> these timers may hurt performance less if the processes would
> > >> hibernate until they get a message. Furthermore, is there any
> > >> documentation explaining why creating many timers can hurt erlang
> > >> performance?
>
> > --------------------------------------------------------------------------
> > > - for hire: mac osx device driver ninja, kernel extensions and usb
> > drivers
>
> > ---------------------+------------+---------------------------------------
> > >http://wagerlabs.com| @wagerlabs |
> >http://www.linkedin.com/in/joelreymont
>
> > ---------------------+------------+---------------------------------------
>
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questi...@REDACTED
> >http://erlang.org/mailman/listinfo/erlang-questions
>
> --
> Best Regards,
> - Ahmed Omarhttp://nl.linkedin.com/in/adiaa
> Follow me on twitter
> @spawn_think <http://twitter.com/#!/spawn_think>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questi...@REDACTED://erlang.org/mailman/listinfo/erlang-questions



More information about the erlang-questions mailing list