[erlang-questions] Newbie - Erlang VS other functional languages

[Fred Hebert]

On 01/16, Lloyd R. Prentice wrote:
> But I struggle with such questions as:
> 
> -- Should each "tool" be implemented as a gen-server?
> -- Should the user interface and database be further factored as separate processes?
> -- Should each "tool" be developed as a separate Erlang application then integrated as dependencies of a higher-level portal? Or should they be developed as a set of modules in one application?
> -- Or maybe each tool should be a totally independent "microservice," whatever that means.
> 
> In other words, my attentive studies of Erlang have left me with very few PRACTICAL architectural techniques and tools; that is an insufficient bridge between the PRINCIPLES of concurrent Erlang and the PRACTICE of building robust Erlang systems.
> 
> I'd much appreciate any guidelines to help me thrash through my confusion.
> 
> But more, I wonder if others struggle with the same issues? And if so, how can I work with wizards like you to shed light on this corner of Erlang technology?
> 

I'd call this the 'zen of Erlang', or just "thinking in Erlang". After a
bit of time, the separation of these things starts coming naturally.

The initial level of confusion with Erlang and OTP, I think, drains its
power in the same confusion as design patterns in OO. You are presented
a bunch of tools and told "here, those tools are great, learn them all."

And then you go on your way to solve problems. And every problem you
meet, you think "which of my tools is the most appropriate for this?"
It's possible you look at some tool and go "well, many of these seem
good! What do I do?"

The problem there is that it's going about it the wrong way. Rather than
seeing a problem and saying "which hammer do I use to smash this?" we
should ask the questions:

- How do I want this to fail? Does it need to take anything down with
  it? How will it come back up? What's my road to recovery?
- What is going to fail? The only correct answer is 'everything, sooner
  or later'. Specifically, things you haven't planned for will crash
  earlier (because you haven't taken the time and attention to plan for
  them).
- The follow-up question is then: how do I make sure I know why
  something failed as clearly as possible? You want to crash early and
  often, but also as close to the source as possible.
- How do I make sure crashes are as limited in their damage as possible?
  The idea is always that you want the damage to be as minimal as
  possible, but you also want *uncertainty* to be as restrained as
  possible. Juggling both ideas mean you will often want to kill related
  things together because you don't want (or haven't taken the time) to
  think about what happens when things that depend on each other fail
  partially. It's often better to kill them at once.
- Who will own and modify that code, and how often will it change? It's
  often a lot easier to throw out code and rewrite it than to update it
  (research says at 25% rewrite, you're better starting from scratch  --
  http://mononcqc.tumblr.com/post/31767374324/an-analysis-of-errors-in-a-reuse-oriented-development-en)
- How will you deploy that code? This is particularly useful if you plan
  on doing hot code loading. Hot code loads will go wrong from time to
  time and reducing the scope of which components you'll impact may be a
  good way to think about it.

Note that none of this is related to Erlang *at all*. It's related to
thinking about your problems, your users, and your team. This is what
really has a great impact on the quality of the final solution. The
technology you use is a ladder. Some are better than others, some can do
wonders with a rope, but an elevator is best. In any case,  the tech
helps you, but you have to use it right.

Once you have thought about this, you will already have made a fairly
good idea of what components should be independent, what kind of
separation you want to have in them, how they depend on each other.

You've done the hardest of the design. Once this is all done, then it's
easier to go "gen_server manages resource", "gen_fsm implements
protocol", "gen_event forwards messages or reacts to events".

You'll probably agree that this doesn't help you much in your questions.
The reason is that what to pick and how to subdivide your system is a
design question, not an Erlang questions. Links, monitors, isolation,
behaviours, and so on are all building blocks, but you're in charge of
assembling them in a cohesive manner.

Understanding what is needed for your problem to be solved is what will
make or break your Erlang design, and it's difficult to apply recipes
wholesale. Or at least, I have found it hard so far in my career.

Regards,
Fred.