[erlang-questions] Structure of a *scalable* application?

[Chris Pacejo]

On Fri, Aug 22, 2014 at 12:24 PM, Michael Loftis <mloftis@REDACTED> wrote:
> Those are all design decisions, none of which will affect your scaling much IMO,

Yes that I realize; I just wonder if there is one "accepted" way.

> but one of your assumptions, only one worker per node, your
> "scaling" would suck,

Sorry, I mean in the context of CPU-boundedness.  I want to keep the
example simple.

By "worker", it may be more general to say "agent within the node
which can accept and distribute work within the node".

> Most erlang applications are designed to run an instance per node of the
> application module, they then RPC to other instances as necessary (see
> mnesia).  I can't think of any offhand where a single application
> instance runs across multiple nodes but I'm sure it exists.

OK, this makes sense.

> Further as to 3) the erlang distribution protocol runs over TCP.  It
> has it's own scaling limitations as some have found out - f/ex if
> you're pushing many megabyte files at a high rate, it might be better
> to have your own RPC mechanism to transfer those rather than sending
> them over erlang distribution directly - the distribution protocol
> comes down to a single thread on a single socket and can become a
> bottleneck for inter-node communications.

Yes this I'm aware of.  The system I'm building is moreso "small
number of nodes which are CPU or I/O-bandwidth-bound", not "large
number of nodes" nor "I/O-latency bound".

> Really though the best advice is to get it right first, then get it
> fast.  Early optimization often isn't nearly as much of a win as
> people think.

Yes I know.  I just want to make sure I structure my application in
such a way that when it comes time to scale, I don't have to refactor
everything.

Thanks!