[erlang-questions] Dynamically add node to Distributed Application

[Paul Mineiro]

If you use this, make sure your ec2 instances are in the same redundancy
zone.  It seemed to make a difference (in ping times and partition event
likelihood).

If the goal is a highly dynamic cluster, I would encourage you to try and
minimize the distribution coupling, and shoot for an architecture of
Erlang nodes "flying in loose formation".  Mnesia in particular is not
really designed for the case of nodes being introduced and removed
frequently (i.e., schemafinder was a bit dangerous for that).  Having
Erlang maintain a fully connected grid at the distribution layer seemed
safe enough for smallish clusters (i.e., nodefinder was useful and caused
no headaches at the scale of < 40 nodes), and rpc:call/5 with a timeout
was a useful primitive, but trying to abstract away local from remote was
something we quickly abandoned.  We also never used the distributed
aspects of the app controller.  We did use pg2 alot, but when starting up
an app we had to run some checks to wait until it was consistent, since
the time it took to settle down was variable.

2c,

-- p


On Fri, 3 Sep 2010, Anthony Molinaro wrote:

> Perhaps this will work for you
>
> http://code.google.com/p/nodefinder/
>
> it has an ec2nodefinder package which will use ec2-describe-instances,
> we used this in conjunction with
>
> http://code.google.com/p/erlrc/
>
> at my previous company to dynamically add ec2 instances and have them
> join our cluster.
>
> We also used
>
> http://code.google.com/p/schemafinder/
>
> to allow distributed mnesia instances to shuttle around data to newly
> created nodes.
>
> -Anthony
>
> On Thu, Sep 02, 2010 at 11:09:29PM +0200, Steve Strong wrote:
> > Hi,
> >
> > I'm working on the design of an Erlang based system which will be hosted on
> > EC2.  Right now, a distributed application with a set of nodes looks like
> > the right way forward, but one of my requirements is to be able to
> > dynamically spin up new amazon instances and add them to the nodes within
> > the distributed app.
> >
> > However, I can't see a way of adding nodes to a running distributed app - I
> > am, however, a self-confessed Erlang noob, so any pointers would be much
> > appreciated!
> >
> > Cheers,
> >
> > Steve
>
> --
> ------------------------------------------------------------------------
> Anthony Molinaro                           <anthonym@REDACTED>
>
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>
>