[erlang-questions] Erlang newbie questions

[Jachym Holecek]

# Gerry Weaver 2011-10-17:
> I am new to Erlang and I'm trying to make the case to continue learning
> this language. I have read all of the books available and seen several
> presentations on the language. I am interested in Erlang primarily for
> the distributed and fault tolerant features it offers.

And you won't be let down on that -- but it takes a while to appreciate
the higher-level engineering aspects of OTP, so don't give up too early.
Given your good theoretical preparations, I'd suggest the next good step
would be to write a non-trivial toy project in Erlang -- after all, it's
a hard fact of nature that actually doing it is the only way to ever
learn anything.

> However, I have
> some reservations about how useful this really is. For example, if you
> have a network server and the listener process fails, how do you recover?

The bright idea is that Erlang/OTP provides you with very natural tools
to address these situations (impenetrable process isolation being the
basis, but you have timeouts for free, links, monitors, etc), but the
exact strategy is inherently application-specific. Popular choice for
dealing with failed communication link would be to schedule reconnect
attempt and revert to hot-spare meanwhile. I suppose MIT X11's mantra,
"mechanism, not policy", applies nicely here (regarding base Erlang/OTP).

> It seems that the fault tolerance is only applicable in the VM context.

Fair observation, it's actually quite reasonable to think of one BEAM
process as an operating system instance of its own -- it happens to run
on top of some UN*X hypervisor typically, and you'd usually arrange for
the BEAM to be restarted on critical failure (much like you'd configure
your OS to automatically restart on kernel panic). Does this make sense?

> This is, in my mind equivalent to any other application process. If I
> have a network server written in C, I can have a watchdog process to kill
> and/or restart it in the event of failure. The C based approach actually
> seems more robust. In this scenario, how would one restart the Erlang VM?

A number of choices -- I know Solaris SMF works, I imagine DJB's supervise
would work too (as exotic as djbware is...). One thing to emphasize here
is that crashing BEAM means one of two things: 1) you didn't provision for
the right kind of flow-control (or rate throttling or ...) mechanisms to
ensure your node fits applicable resource limits -- for this you can only
blame yourself, or 2) you somehow managed to provoke a drastic error in
BEAM (very very very rare, but not unheard of). Either case is really
akin to an OS crash -- a catastrophic, worst-case scenario. Of course,
other nodes talking to the failed one can deal with it gracefully.

> It has been my experience that network servers most often fail due to OS
> or hardware limits rather than bugs in the code

See above, hitting OS/HW limits is just engineer's failure to exert due
diligence to fit within those limits. Hardware failure or operator error
however I couldn't argue with. Also, see below on completely different
scales of software integration you'll meet in Erlang -- that increases
the likelihood of programming error wanting to spoil the party.

> (at least for thoroughly tested production ready code).

[Must suppress the urge to make cynical comments.]

> When you factor in the amount of baggage
> that comes with Erlang, or any other VM based language, it's difficult
> to justify.

I can understand this prejudice, other way to see this is that, much like
an OS, the BEAM is part of your TCB. Any error therein is fatal and you
can't do much about it. One of the services it provides you however is
blank cheque on safety one Erlang process has regardless, for one, of
what other Erlang processes do, but for other regardless of what the
code running in that process does (is allowed to do). The guarantess
given to a UN*X process by OS kernel seem rather lame in comparison.
In my mind, this is definitely worth it.

> Now there is also the rapid development aspect, but I'm
> finding that at least for now, the time savings is being eaten up by
> looking for documentation. I understand that this situation will improve
> over time, but the various posts I've seen from folks with more experience
> seem to indicate that this will take quite a while. I like the language
> (except the , ; . thing).

Yep, there's some learning curve to deal with, but the closer it starts
resembling a constant function, the more unbelievable will your productivity
be -- rolling production-ready solutions once every three weeks is not
complete science-fiction :-). One thing I wanted to mention, and this
is rather random place to do so, is that besides the other advantages
of Erlang/OTP you've already mentioned (or the books you've read surely
did) there's also one people don't talk about very often: high-level
compositionality of sorts -- not at the level of functions or modules,
but whole applications (or application stacks). You have an existing
solution and suddenly are tasked with adding Diameter interface for
charging, a web-based GUI for Operations and SNMP monitoring interface
all at once? Big deal, just add a few protocol stacks to your boot
file, write a few glue-modules exporting existing information to them,
and you're good to go.

> You can do some pretty cool things with it,
> but when I got over that initial gee whiz, I had some trouble seeing a
> real world production use case.

I'd choose Erlang/OTP for anything that requires complex control logic
or gluing together of many networked systems. I'm a bit too tired to
give specific examples at the moment, can fill in tomorrow and others
will surely be eager to provide their own if you're interested.

> I'm not trying to be critical, I just
> figure I must have missed something along the way. Any perspectives or
> advice on this would be greatly appreciated.

That's fine, coming from embedded systems background (the land of C and
asm) I can understand your skepticism... (I know I wouldn't have believed
nice words until I 1) was open-minded enough to listen 2) went on writing
Erlang code for a while).

Hope this slightly wine-infused mail was helpful, or at least completely
harmless. :-)

BR,
	-- Jachym