[erlang-questions] beginner architectural question

[Garrett Smith]

Hi Semmit,

On Tue, Nov 11, 2014 at 3:24 AM, semmit mondo <semmitmondo@REDACTED> wrote:
> Hi,
>
> I have an application with a structural pattern that comes up again and
> again, and I would like to ask you how would you solve this.  My app has
> two kinds of processes:  a singleton that is a server collecting data, and
> workers that do some calculation and send the result to the server.
> There can be any number of workers and there's usually one data
> collector server.  I think this is a very simple and common architecture.
> The problem with this is that the calculators have to know the pid of the
> collector and the whole thing has to be supervised.

They don't have to know the pid. That's what the registry is for.

> The simplest solution is that under a main supervisor I spawn the collector
> server and register it with a name.  Next to the collector I start another
> supervisor that manages the pool of the calculators and those processes
> simply use the collectors name as the message destination when they're
> done with the jod.  The main supervisor uses one_for_one strategy, while
> the subsupervisor is a simple_one_for_one.

Exactly right.

But you actually never spawn/start anything - you specify the child
config for the supervisor and it starts the processes.

The simple_one_for_one supervisor to start collectors.

You should never call spawn/start_link for any of these yourself.

> I feel that this is not the best I could do.  Another solution could be to
> use rest_for_one in the top supervisor, not register a name for the
> collector, but push down the pid of the top supervisor to the sub-
> supervisor, and then in the init function of the sub-supervisor I can
> ask the top supervisor (with a which_children() call) to give me the
> pid of the collector.  So I can give it to the calculators as an argument.
> Seems a bit difficult, but has the benefit not using a registered name
> for a purely internal use case.

I don't understand the problem you're trying to solve here.

The process registry is the canonical way to avoid the pid lookup
problem. You wrap this so called "singleton" in a facade module (API)
and just make calls without any thought to where they'll be sent.
Internally of course, you use the process name (typically the name of
the module that implements the process behavior) to your calls to
gen_server:cast/call.

> There's also a wrong solution with a top supervisor with no static
> children.  In that case I can start the children manually.  First I can
> start the collector, then I have got the pid of it and can use it when
> starting the calculators.  The problem here is that when the collector
> gets restarted, the calculators won't have the right pid.  Even if the
> top supervisor uses rest_for_one strategy, and the calculators get
> restarted, there's no mechanism to start calculators with the new
> pid of the collector.  (I can't give the supervisor a function that
> describes the way how to restart the whole thing, can I?)

You are making an excellent case for the process registry. Try it!

> This wrong approach can be fixed if the top level supervisor exits
> when the collector or the sub-supervisor fails (that's a {one_for_one,
> 0, 1} supervisor setup) and a plus layer of supervision above this
> supervisor gets into play.  But that's just too many layers and
> supervisors for a simple thing like this, am I right?

My eyes have glazed over - I think you're solving problems of your own
imaginings. Use the process registry.

> I believe that the whole thing boils down to that I have not much
> control over the order of child starts and I can't move information
> between children of the same supervisor.  It would be nice if a
> supervisor could supply some sort of a local name registry for the
> processes below them but as far as I know there's only a global
> (node local) one that could be used here.

Don't try to wire these things up - you'll drive yourself crazy.
Create an independent gen_server (named/registered) - you can think of
this as a singleton. I prefer to think of it as a "service". This
thing hides behind a module interface that doesn't reveal anything
about its implementation (i.e. whether it's a process, how the process
is looked up, etc.)

Run this collector service under the root supervisor. Let it go. Trust
it. It'll be fine.

Create a simple_one_for_one supervisor, also managed by the root. This
guy is a "factory" for the calculators/workers. Provide a
"start_calculator" function that calls the supervisor's start_child
function.

Each calculator does it's thing and uses your collector module
(facade) to report back data.

> Which architecture would you choose?  Are there other alternative
> structures I haven't thought of?  What do you thing which one is the
> best approach and why?

Keep it simple. Run your processes under supervision.

The singleton/server pattern is straight forward - use a
named/registered process and provide its functionality via a module
that doesn't require any server pid. Super simple.

The worker pattern is straight forward - use a simple one for one
supervisor to start the children. The simplest model is to "fire and
forget" with these things.

Try this and see what happens. Fix problems that you encounter - not
problems that you think you might encounter or feel or believe.

Garrett