[erlang-questions] pg2 scale limits

Michael Truog <>
Thu Dec 12 19:50:40 CET 2013


If you want to avoid the global lock, you can do so with cpg (https://github.com/okeuday/cpg/), which provides the same pg2 API with additional functionality.  It does this with the default macro define of GROUP_NAME_WITH_LOCAL_PIDS_ONLYwhere the global lock is not needed if only local Erlang pids are used with the cpg API.  If necessary, the define could be commented in the cpg_constants.hrl file, to have the same global locking pg2 does.  cpg doesn't use ets like pg2 does, and with the default cpg_constants.hrl defines it requires the group names are strings (list of integers), but that can be changed with GROUP_STORAGE.

On 12/12/2013 09:30 AM, Matthew Evans wrote:
>
> The one thing to watch for is that pg2 will take a global lock over all nodes in a mesh when a join/leave is done. This means that if you are doing lots of joins, and have a mesh of Erlang nodes, then there is a performance overhead. Not a massive one, but something to watch for.
>
> > From: 
> > To: 
> > Date: Wed, 11 Dec 2013 10:19:42 -0600
> > Subject: [erlang-questions] pg2 scale limits
> >
> > What has been the experience of folks with the practical upper bound of
> > process groups registered with the pg2 application?
> >
> > I have an application where I need to track 10000 or so process groups
> > across a set of erlang nodes (2 R14B04 Erlang VMs running on top of 2
> > Debian VMs to start.) Would that size of registry cause problems?
> >
> > I read
> > http://christophermeiklejohn.com/erlang/2013/06/03/erlang-pg2-failure-seman
> > tics.html and it was a great read - are there any other pitfalls about pg2
> > that I should be aware of?
> >
> > I am planning to implement a "prototype" of this application using pg2 at
> > reasonably low scale, but I am open to other implementation suggestions or
> > applications (I looked at gproc too, but I'd rather stick with pg2 if
> > possible because it comes "out of the box"...)
> >
> > Thanks.
> >
> > Mark
> >
> > _______________________________________________
> > erlang-questions mailing list
> > 
> > http://erlang.org/mailman/listinfo/erlang-questions
>
>
> _______________________________________________
> erlang-questions mailing list
> 
> http://erlang.org/mailman/listinfo/erlang-questions

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20131212/74fb827d/attachment.html>


More information about the erlang-questions mailing list