[erlang-questions] gen_server and multiple workers

[Paul Mineiro]

The document is a little ambiguous wrt whether load pids are load
balanced.  Fortunately the source is not.

-------
pmineiro@REDACTED% grep -n -A15 'get_closest_pid(Name)' /sw/lib/erlang/lib/kernel-2.11.5/src/pg2.erl
122:get_closest_pid(Name) ->
123-    case get_local_members(Name) of
124-    [Pid] ->
125-        Pid;
126-    [] ->
127-        {_,_,X} = erlang:now(),
128-        case get_members(Name) of
129-            [] -> {error, {no_process, Name}};
130-            Members ->
131-                lists:nth((X rem length(Members))+1, Members)
132-        end;
133-    Members when is_list(Members) ->
134-        {_,_,X} = erlang:now(),
135-        lists:nth((X rem length(Members))+1, Members);
136-    Else ->
137-        Else
-------

line 135 indicates randomization.

-- p


On Thu, 30 Aug 2007, David King wrote:

> > You could create multiple gen_servers and instead of registering
> > their pids (using start_link/4) instead keep them anonymous (using
> > start_link/3) and then have them joined a named process group (using
> > pg2:join/2) and then issue requests using pg2:get_closest_pid/1.
>
> That sounds like exactly what I want. Does pg2:get_closest_pid/1
> handle balancing, or will it return the same PID every time it's
> called from a given machine? (The documentation is not clear on
> this.) Is a named process group accessible from any machine in a
> cluster?
>
> Thanks for getting back to me so quickly, by the way!
>
> > On Thu, 30 Aug 2007, David King wrote:
> >
> >> I imagine that this is a problem that has been solved a hundred times
> >> and that I just don't have the right search terms :)
> >>
> >> I need to communicate with an outside program, for which I'm using a
> >> port and a gen_server. However, I may be getting many requests for
> >> communication to this port at a time, and I don't want to introduce a
> >> bottleneck in the gen_server, nor do I want to fork() a new instance
> >> of the program for every request. So I think I'd like to have several
> >> workers, where each request grabs an available worker (blocking until
> >> one is available), ideally with some place to put logic to expand or
> >> reduce the size of the worker-pool with load, but that's optional.
> >> Having several copies of the external program open at a time isn't a
> >> problem (it's just a text-transform done by a Perl program).
> >>
> >> My gen_server doesn't keep any state except for the Port, so that
> >> could be managed another way. It seems that I can't combine this with
> >> gen_server, as gen_server seems to want to register() its PID, and I
> >> can't have multiple workers with the same register()ed name.
> >>
> >> I have a supervisor watching the gen_server. supervisor's required
> >> export init/1 returns (among other things) a list of processes to
> >> enter the supervision, but it just calls start_link on the
> >> gen_server, which registers its name.
> >>
> >> I could have the worker-pool managed by the single gen_server
> >> instance, but I'd like them to be able to crash independently, and it
> >> seems that that would complicate the handle_call function (as
> >> gen_server seems to assume that it's synchronous). Complication is
> >> okay, but I'd like to avoid it if it turns out that there is a single
> >> library that already does what I want :)
> >>
> >> I could also have the supervisor pass in a different name to register
> >> to gen_server:start_link (worker1,worker2, ...) and have the
> >> supervisor manage the pool, but that seems messy (which, again, is
> >> okay, but is to be avoided if possible).
> >>
> >> Anyway, I'm sure this is a solved problem and I'm just looking in the
> >> wrong places. How would you do this?
>

"A hot dog and bun, you have to have a style and strategy that's
different from a chicken wing, which is different from a matzo ball,"
he says. "Athletics are not really about superior fitness. They're about
superior refinement of skill. That's what Babe Ruth did. That's what
this is."

http://en.wikipedia.org/wiki/Competitive_eating