[erlang-questions] Erlang newbie questions

[Joe Armstrong]

On Tue, Oct 18, 2011 at 10:32 AM, CGS <cgsmcmlxxv@REDACTED> wrote:
> Hi,
>
> Interesting discussion here and I would like few points (as C is also my
> favorite programming language).
>
> The main point for entering the discussion is the fact that many who replied
> here were saying that if something crashes in your application doesn't kill
> your application like in C. I beg to differ on this point. If your Erlang
> application is single threaded, then your application (I do not speak here
> about OTP application) will crash if an error occurs there. If you construct
> your C application like an OTP application, then that means your C
> applications is multithreaded, so, if one thread dies, it doesn't crash your
> main application (as well as in OTP application, if one gen_server crashes
> the whole application doesn't crash).

Right but in order to "construct your C application like an OTP
application" means
you will have to re-invent 50% (at a conservative estimate) of the
Erlang run-time.

The deep problem is not making your C application mutithreaded - it's making
it fault-resilient. Mutithreaded C implies some kind of data exchange between
different C threads - now there are many ways to do this - the
cleanest alternative
is by copying message passing. The dirtiest - is by shared memory and mutexes.

Shared memory and mutexes is not where you want to go (willingly).

The deep problem has a deeper problem hiding inside it - memory
fragmentation and reallocation
of pointers. This implies some kind of GC or some kind of static
analysis that is beyond the brain of man
or beast to comprehend.

So what do you do? - you decide to not share memory and let each
process have it's own stack and heap
and you copy everything. Now you've invented 15% of Erlang and are
approximately where we were in 1986.

The tricky bits are deep under the surface - addressing memory
fragmentation and failure. Now simple
things don't need all this stuff - so if you want to write an embedded
washing machine controller write it in C.
But if you want to write a few millions of lines of complex code don't
write it in C. Write the 5% of messy stuff that needs to be fast in C
and write the other 95% in Erlang.

(( This is exactly what Dennis Richie (RIP) said about C and assembler
only with the 5% in assembler))

The productivity gain comes from the fact that it is possible to write
95% of a system in {Erlang, C}
and 5% in {C, Assembler}

Just as in the early days of C some systems should be written entirely
in Assembler, today some systems should be written entirely in C.

What Erlang does is move the goal-posts.

If you're only writing small systems - you won't notice this and C is
fine. But if you want to write systems that run
forever and have a large number of lines of code use Erlang. Try to
write it all in Erlang, if this fails write selected parts in C.

This is actually what you would do in C. Write it all in C and then if
this fails write selected parts in assembler.

All we've done is move the goal posts.

Cheers

/Joe
>
> Now, Gerry, think Erlang as being multithread application in C where you
> don't need to care too much about how to construct the thread. That means,
> the construction of each thread is much simpler and you already have the
> monitor (watchdog, if you prefer) part implemented, so, you can focus on
> what you want to develop and less on the tools you need.
>
> There are few more aspects here you may want to consider before choosing in
> which language you want to write your code.
>
> Firstly, the network applications. In C, due to the lack of standards, you
> have to know the compiler in which you want to program a port listener (as
> example). In Erlang, you get the standard which is missing in C. Therefore,
> the productivity is higher in case of Erlang in this case.
>
> Staying in the field of networking, some may argue that Erlang is not the
> only one to be productive here. And that is correct if you consider JAVA (as
> one of the main languages capable of creating cross-platform network
> applications). Still, I would prefer Erlang for its VM and basic concepts,
> and, as C programmer, I think you would understand me why I prefer each
> thread to have its own memory (just for example).
>
> Secondly, as desktop application, generally, Erlang is far worse choice than
> C. Still, if you don't care too much of the speed of your application,
> Erlang offers some nice tools for increasing your productivity as
> programmer. Also, due to its capability of interacting with external
> functions written in C, you can create your own fast libraries which you can
> connect them to Erlang (here you will spend a little bit of time to
> understand the concept behind).
>
> Thirdly, concerning the application design. If in C you need to design your
> application from A to Z, in Erlang you have tools which limit your design,
> but it can increase your overall productivity. Actually, I consider OTP
> design as being one of the most useful for general purpose.
>
> I will stop here with the list. You will discover many other advantages
> using Erlang during making your own experience with Erlang. I am no expert
> in Erlang and many things I am still discovering while programming in this
> language. And, for you to see that I bumped into the same problems, I had
> also some headaches with the line separators (many times I got compiling
> errors due to them). But, in the end, those separators turned out to be more
> useful than pain in the... Just for example, a loop in C when you don't want
> to use the for statement is written like this (take it as principle and not
> as something to be compiled):
>
> myfunc(myvars)
> {
>  exit_condition;
> the_loop_body;
> myfunc(modified_vars);
> }
>
> while in Erlang, it is looks like that:
>
> myfunc(myvars) ->
> the_loop_body,
> myfunct(modified_vars);
>
> myfunc(stop_vars) ->
> end_of_loop.
>
> As you can see it's not so different, but you have to implement explicitly
> the end of the loop. Quite a pain you may say. Actually, using this method,
> think of having two functions with the same name, but with the language
> choosing which one to be used depending on what arguments you pass to it.
> Doesn't it sound a bit familiar? Yes. And more than that, you know the
> advantage of this in practical applications. So, "two rabbits with a single
> gun shot" (as there is a saying in my country).
>
> OK! One can speak whole day long about the advantages and disadvantages of
> Erlang, but the most important is for you to make your own experience. So
> far, my experience with Erlang was positive and I found myself liking Erlang
> quite some. I wish your experience with Erlang to be as good as mine. Just
> don't forget each language has its own beauty, but you need to know from
> which angle you should look at it.
>
> Cheers,
> CGS
>
>
>
>
> On 10/18/2011 02:17 AM, Gerry Weaver wrote:
>>
>> Hi,
>>
>> Wow! I really appreciate your feedback. I find your responses very logical
>> and helpful ;-)
>>
>> Another issue that I've started to think about is deployment. There seem
>> to be several competing tools/utilities for this. Is there a preferred or
>> recommended solution?
>>
>>
>> Is the Erlang language itself still evolving? The reason I ask is that the
>> fellow that did CouchDB posted a blog entry about the things he encountered
>> while developing CouchDB. What was interesting to me is how, even after a
>> short time with the language, many of the issues he mentioned immediately
>> rang true with me. For example, what is the benefit of the "; , ."
>> terminators as opposed to a single terminator approach?
>>
>>
>> I don't mean to sound negative here. I'm just poking around a bit. I find
>> Erlang quite compelling overall. I'm digging into the OTP at this point and
>> there are some really interesting capabilities there.
>>
>> I assume a best practice is to minimize the use of modules written in
>> other languages. What exactly would the scenario be for a C module that
>> segfaults? I assume it would take down the VM.
>>
>>
>>
>>
>> Thanks,
>> -G
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> -----Original Message-----
>>>
>>> From: "Jachym Holecek"<freza@REDACTED>
>>> To: "Gerry Weaver"<gerryw@REDACTED>
>>> Cc: erlang-questions@REDACTED
>>> Date: 10/17/11 17:41
>>> Subject: Re: [erlang-questions] Erlang newbie questions
>>>
>>> # Gerry Weaver 2011-10-17:
>>>>
>>>> I am new to Erlang and I'm trying to make the case to continue learning
>>>> this language. I have read all of the books available and seen several
>>>> presentations on the language. I am interested in Erlang primarily for
>>>> the distributed and fault tolerant features it offers.
>>>
>>> And you won't be let down on that -- but it takes a while to appreciate
>>> the higher-level engineering aspects of OTP, so don't give up too early.
>>> Given your good theoretical preparations, I'd suggest the next good step
>>> would be to write a non-trivial toy project in Erlang -- after all, it's
>>> a hard fact of nature that actually doing it is the only way to ever
>>> learn anything.
>>>
>>>> However, I have
>>>> some reservations about how useful this really is. For example, if you
>>>> have a network server and the listener process fails, how do you
>>>> recover?
>>>
>>> The bright idea is that Erlang/OTP provides you with very natural tools
>>> to address these situations (impenetrable process isolation being the
>>> basis, but you have timeouts for free, links, monitors, etc), but the
>>> exact strategy is inherently application-specific. Popular choice for
>>> dealing with failed communication link would be to schedule reconnect
>>> attempt and revert to hot-spare meanwhile. I suppose MIT X11's mantra,
>>> "mechanism, not policy", applies nicely here (regarding base Erlang/OTP).
>>>
>>>> It seems that the fault tolerance is only applicable in the VM context.
>>>
>>> Fair observation, it's actually quite reasonable to think of one BEAM
>>> process as an operating system instance of its own -- it happens to run
>>> on top of some UN*X hypervisor typically, and you'd usually arrange for
>>> the BEAM to be restarted on critical failure (much like you'd configure
>>> your OS to automatically restart on kernel panic). Does this make sense?
>>>
>>>> This is, in my mind equivalent to any other application process. If I
>>>> have a network server written in C, I can have a watchdog process to
>>>> kill
>>>> and/or restart it in the event of failure. The C based approach actually
>>>> seems more robust. In this scenario, how would one restart the Erlang
>>>> VM?
>>>
>>> A number of choices -- I know Solaris SMF works, I imagine DJB's
>>> supervise
>>> would work too (as exotic as djbware is...). One thing to emphasize here
>>> is that crashing BEAM means one of two things: 1) you didn't provision
>>> for
>>> the right kind of flow-control (or rate throttling or ...) mechanisms to
>>> ensure your node fits applicable resource limits -- for this you can only
>>> blame yourself, or 2) you somehow managed to provoke a drastic error in
>>> BEAM (very very very rare, but not unheard of). Either case is really
>>> akin to an OS crash -- a catastrophic, worst-case scenario. Of course,
>>> other nodes talking to the failed one can deal with it gracefully.
>>>
>>>> It has been my experience that network servers most often fail due to OS
>>>> or hardware limits rather than bugs in the code
>>>
>>> See above, hitting OS/HW limits is just engineer's failure to exert due
>>> diligence to fit within those limits. Hardware failure or operator error
>>> however I couldn't argue with. Also, see below on completely different
>>> scales of software integration you'll meet in Erlang -- that increases
>>> the likelihood of programming error wanting to spoil the party.
>>>
>>>> (at least for thoroughly tested production ready code).
>>>
>>> [Must suppress the urge to make cynical comments.]
>>>
>>>> When you factor in the amount of baggage
>>>> that comes with Erlang, or any other VM based language, it's difficult
>>>> to justify.
>>>
>>> I can understand this prejudice, other way to see this is that, much like
>>> an OS, the BEAM is part of your TCB. Any error therein is fatal and you
>>> can't do much about it. One of the services it provides you however is
>>> blank cheque on safety one Erlang process has regardless, for one, of
>>> what other Erlang processes do, but for other regardless of what the
>>> code running in that process does (is allowed to do). The guarantess
>>> given to a UN*X process by OS kernel seem rather lame in comparison.
>>> In my mind, this is definitely worth it.
>>>
>>>> Now there is also the rapid development aspect, but I'm
>>>> finding that at least for now, the time savings is being eaten up by
>>>> looking for documentation. I understand that this situation will improve
>>>> over time, but the various posts I've seen from folks with more
>>>> experience
>>>> seem to indicate that this will take quite a while. I like the language
>>>> (except the , ; . thing).
>>>
>>> Yep, there's some learning curve to deal with, but the closer it starts
>>> resembling a constant function, the more unbelievable will your
>>> productivity
>>> be -- rolling production-ready solutions once every three weeks is not
>>> complete science-fiction :-). One thing I wanted to mention, and this
>>> is rather random place to do so, is that besides the other advantages
>>> of Erlang/OTP you've already mentioned (or the books you've read surely
>>> did) there's also one people don't talk about very often: high-level
>>> compositionality of sorts -- not at the level of functions or modules,
>>> but whole applications (or application stacks). You have an existing
>>> solution and suddenly are tasked with adding Diameter interface for
>>> charging, a web-based GUI for Operations and SNMP monitoring interface
>>> all at once? Big deal, just add a few protocol stacks to your boot
>>> file, write a few glue-modules exporting existing information to them,
>>> and you're good to go.
>>>
>>>> You can do some pretty cool things with it,
>>>> but when I got over that initial gee whiz, I had some trouble seeing a
>>>> real world production use case.
>>>
>>> I'd choose Erlang/OTP for anything that requires complex control logic
>>> or gluing together of many networked systems. I'm a bit too tired to
>>> give specific examples at the moment, can fill in tomorrow and others
>>> will surely be eager to provide their own if you're interested.
>>>
>>>> I'm not trying to be critical, I just
>>>> figure I must have missed something along the way. Any perspectives or
>>>> advice on this would be greatly appreciated.
>>>
>>> That's fine, coming from embedded systems background (the land of C and
>>> asm) I can understand your skepticism... (I know I wouldn't have believed
>>> nice words until I 1) was open-minded enough to listen 2) went on writing
>>> Erlang code for a while).
>>>
>>> Hope this slightly wine-infused mail was helpful, or at least completely
>>> harmless. :-)
>>>
>>> BR,
>>>      -- Jachym
>>>
>>>
>>>
>>>
>>
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>