[erlang-questions] gen_server and multiple workers

Paul Mineiro paul-trapexit@REDACTED
Thu Aug 30 19:46:21 CEST 2007

The document is a little ambiguous wrt whether load pids are load
balanced.  Fortunately the source is not.

pmineiro@REDACTED% grep -n -A15 'get_closest_pid(Name)' /sw/lib/erlang/lib/kernel-2.11.5/src/pg2.erl
122:get_closest_pid(Name) ->
123-    case get_local_members(Name) of
124-    [Pid] ->
125-        Pid;
126-    [] ->
127-        {_,_,X} = erlang:now(),
128-        case get_members(Name) of
129-            [] -> {error, {no_process, Name}};
130-            Members ->
131-                lists:nth((X rem length(Members))+1, Members)
132-        end;
133-    Members when is_list(Members) ->
134-        {_,_,X} = erlang:now(),
135-        lists:nth((X rem length(Members))+1, Members);
136-    Else ->
137-        Else

line 135 indicates randomization.

-- p

On Thu, 30 Aug 2007, David King wrote:

> > You could create multiple gen_servers and instead of registering
> > their pids (using start_link/4) instead keep them anonymous (using
> > start_link/3) and then have them joined a named process group (using
> > pg2:join/2) and then issue requests using pg2:get_closest_pid/1.
> That sounds like exactly what I want. Does pg2:get_closest_pid/1
> handle balancing, or will it return the same PID every time it's
> called from a given machine? (The documentation is not clear on
> this.) Is a named process group accessible from any machine in a
> cluster?
> Thanks for getting back to me so quickly, by the way!
> > On Thu, 30 Aug 2007, David King wrote:
> >
> >> I imagine that this is a problem that has been solved a hundred times
> >> and that I just don't have the right search terms :)
> >>
> >> I need to communicate with an outside program, for which I'm using a
> >> port and a gen_server. However, I may be getting many requests for
> >> communication to this port at a time, and I don't want to introduce a
> >> bottleneck in the gen_server, nor do I want to fork() a new instance
> >> of the program for every request. So I think I'd like to have several
> >> workers, where each request grabs an available worker (blocking until
> >> one is available), ideally with some place to put logic to expand or
> >> reduce the size of the worker-pool with load, but that's optional.
> >> Having several copies of the external program open at a time isn't a
> >> problem (it's just a text-transform done by a Perl program).
> >>
> >> My gen_server doesn't keep any state except for the Port, so that
> >> could be managed another way. It seems that I can't combine this with
> >> gen_server, as gen_server seems to want to register() its PID, and I
> >> can't have multiple workers with the same register()ed name.
> >>
> >> I have a supervisor watching the gen_server. supervisor's required
> >> export init/1 returns (among other things) a list of processes to
> >> enter the supervision, but it just calls start_link on the
> >> gen_server, which registers its name.
> >>
> >> I could have the worker-pool managed by the single gen_server
> >> instance, but I'd like them to be able to crash independently, and it
> >> seems that that would complicate the handle_call function (as
> >> gen_server seems to assume that it's synchronous). Complication is
> >> okay, but I'd like to avoid it if it turns out that there is a single
> >> library that already does what I want :)
> >>
> >> I could also have the supervisor pass in a different name to register
> >> to gen_server:start_link (worker1,worker2, ...) and have the
> >> supervisor manage the pool, but that seems messy (which, again, is
> >> okay, but is to be avoided if possible).
> >>
> >> Anyway, I'm sure this is a solved problem and I'm just looking in the
> >> wrong places. How would you do this?

"A hot dog and bun, you have to have a style and strategy that's
different from a chicken wing, which is different from a matzo ball,"
he says. "Athletics are not really about superior fitness. They're about
superior refinement of skill. That's what Babe Ruth did. That's what
this is."


More information about the erlang-questions mailing list