[erlang-questions] gen_server and multiple workers

[David King]

Thank you Paul for your advice; I was able to get this working. I  
wrote a generic implementation of an N-worker supervisor/gen_server  
that uses an external (Perl) program, and a how-to with the relevent  
code (http://www.ketralnis.com/roller/dking/entry/20070903) if anyone  
wants a copy of it.

On 30 Aug 2007, at 10:09, David King wrote:

>> You could create multiple gen_servers and instead of registering
>> their pids (using start_link/4) instead keep them anonymous (using
>> start_link/3) and then have them joined a named process group (using
>> pg2:join/2) and then issue requests using pg2:get_closest_pid/1.
>
> That sounds like exactly what I want. Does pg2:get_closest_pid/1
> handle balancing, or will it return the same PID every time it's
> called from a given machine? (The documentation is not clear on
> this.) Is a named process group accessible from any machine in a
> cluster?
>
> Thanks for getting back to me so quickly, by the way!
>
>> On Thu, 30 Aug 2007, David King wrote:
>>
>>> I imagine that this is a problem that has been solved a hundred  
>>> times
>>> and that I just don't have the right search terms :)
>>>
>>> I need to communicate with an outside program, for which I'm using a
>>> port and a gen_server. However, I may be getting many requests for
>>> communication to this port at a time, and I don't want to  
>>> introduce a
>>> bottleneck in the gen_server, nor do I want to fork() a new instance
>>> of the program for every request. So I think I'd like to have  
>>> several
>>> workers, where each request grabs an available worker (blocking  
>>> until
>>> one is available), ideally with some place to put logic to expand or
>>> reduce the size of the worker-pool with load, but that's optional.
>>> Having several copies of the external program open at a time isn't a
>>> problem (it's just a text-transform done by a Perl program).
>>>
>>> My gen_server doesn't keep any state except for the Port, so that
>>> could be managed another way. It seems that I can't combine this  
>>> with
>>> gen_server, as gen_server seems to want to register() its PID, and I
>>> can't have multiple workers with the same register()ed name.
>>>
>>> I have a supervisor watching the gen_server. supervisor's required
>>> export init/1 returns (among other things) a list of processes to
>>> enter the supervision, but it just calls start_link on the
>>> gen_server, which registers its name.
>>>
>>> I could have the worker-pool managed by the single gen_server
>>> instance, but I'd like them to be able to crash independently,  
>>> and it
>>> seems that that would complicate the handle_call function (as
>>> gen_server seems to assume that it's synchronous). Complication is
>>> okay, but I'd like to avoid it if it turns out that there is a  
>>> single
>>> library that already does what I want :)
>>>
>>> I could also have the supervisor pass in a different name to  
>>> register
>>> to gen_server:start_link (worker1,worker2, ...) and have the
>>> supervisor manage the pool, but that seems messy (which, again, is
>>> okay, but is to be avoided if possible).
>>>
>>> Anyway, I'm sure this is a solved problem and I'm just looking in  
>>> the
>>> wrong places. How would you do this?
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions