[erlang-questions] Distributed Erlang Architecture

[Garrett Smith]

Hi Chris,

On Wed, May 20, 2015 at 2:02 PM, Chris Clark <boozelclark@REDACTED> wrote:
> Hi
>
> I have an erlang application that is made up of 3 applications and their
> dependencies. I am trying to decide on how to distribute these so that I can
> scale them by running additional instance of which ever app start to reach
> its performance limits.
>
> I've managed to find lot of info of how to connect erlang nodes into a
> cluster but nothing on how to structure a distributed application in
> practice. Does anyone know where I can find some info on this?
>
> If for example I have three hosts in my cluster. Should I
> A:
> Compile these all into one release and the not start the apps automatically
> but just start the number of required instances within the single erlang
> nodes each running on a host

A release corresponds to a node and what's in that release should be
driven by the behavior of the node - there may be some linkage with so
called scalability problems in specific cases, but certainly not in
the general case.

As for not starting "all" - this sounds like a misunderstanding - on
your part or my part. Maybe I'm misreading you but it sounds like
you're viewing your "apps" as units of power - like octane - that you
might want to hold in reserve for the time you need to go super fast.

An app is just a single supervisory tree that provides some set of
functionality within your system (i.e. a running release).

If an "app" in this case can "scale" - then you ought to know how. Is
the constraint CPU, disk, IO - or a host of complex interactions that
you'd never guess in a million years? If and when you understand how
it can scale, look to deploy multiple instances of it per unit of
actual power (CPU, spindle, network interface, etc) - if that would
even work. Chances are you'll have to rethink something fundamental in
the way you're doing things. That's okay though - it's the sign of a
system evolving under pressure.

Erlang is outstanding for this.

> B:
> Create each app into a separate release and then just start a node for each
> instance of each app I want resulting in multiple erlang nodes per host. If
> for example I just wanted one host then I would run three nodes on that
> host. One for each app.

Releases are just ways to package apps and run them in concert in a
VM. If you want one app running on a VM, then do this. Otherwise don't
do this.

> Any guidance or info on the pros and cons of each approach would be
> appreciated.

Knowing nothing about your application, I'd just start by creating one
release with a complete working system (user facing functionality).
Then see how it behaves at various levels of load (request frequency,
concurrency, volumes, etc.)

Personally, I wouldn't think about scalability at all - and not just
to make a social statement. I'd put the system into production and see
what happens. You'll need to monitor things, but start by monitoring
things from the end user perspective - this will force you to define
meaningful performance targets. When you start to see something bad
happening, study it carefully and take steps to fix the problem.

When you're at the point you have actual problems (rather than
architectural, which is not an actual thing, much less a problem) you
can post some detailed information here and get some actual help.

Hope that helps ;)

Garrett