[erlang-questions] trouble with erlang or erlang is a ghetto

[Richard O'Keefe]

On 27/07/2011, at 9:05 AM, Loïc Hoguin wrote:

> On 07/26/2011 09:07 PM, Joel Reymont wrote:
>> Did I miss a lively and heated discussion?
>> 
>> http://www.unlimitednovelty.com/2011/07/trouble-with-erlang-or-erlang-is-ghetto.html
>> 
>> Bring it on!

There are some substantive issues in that blog entry.

(1) Frames have not been implemented yet.

    Now if I had provided a model implementation, things might have been
    different.  Or if Joe had provided a model implementation of his
    earlier and fundamentally similar "proper structs", again things
    might have been different.

    My excuse is that the BEAM architecture is simply not documented
    anywhere that I can find, and I have too many other things to do 
    to grovel around in the guts in Erlang to figure it out.  So let
    me add my item here:

(2) BEAM has no usable documentation.

    One reason this is important is because some people have liked the
    ideas underneath Erlang well enough to try to build other syntaxes
    for it.  But that is inexcusably hard with BEAM undocumented.

(3) There doesn't seem to be anything in the Erlang _approach_ that
    should interfere with scaling, but the _implementation_ appears
    not to scale well much past 16 cores.

    I don't know if that is current information.  If true, it's an
    important limitation of the implementation which I'm sure will be
    given a lot of attention.  I don't expect it to stay true.

(4) He doesn't seem to like the Erlang garbage collector, but that's
    something which has changed more than once, and he does not
    offer any actual _measurements_.

    I tried the experiment of allocating 1,000,000,000 list cells
    (but never keeping more than 10,000 of them at a time).
    Erlang, byte-codes:  7.58 seconds (=  7.58 nsec/allocation).
    Erlang, native    :  3.92 seconds (=  3.92 nsec/allocation).
    Java -O -server   : 11.40 seconds (= 11.40 nsec/allocation).
    Java -O -client   : 12.26 seconds (= 12.26 nsec/allocation).

    Java has come a long way.  I don't have an Azul system to try.

    He praised tcmalloc.  I note that it only recently became usable
    without pain on my laptop (MacOS X) and that building it produced
    reams of warning messages about using a deprecated interface, so
    it may not work much longer.  It doesn't work at all on the other
    machine on my desk.  (Erlang works on both.)  I wrote a similar
    benchmark in C and linked it with libtcmalloc.a.  I killed that
    program after it had run for more than 10 times as long as the
    Erlang code.  So when he says "Erlang ... can't take advantage of
    libraries like tcmalloc", there doesn't appear to be any
    advantage that Erlang *could* take.

    In short, Erlang's memory management is criticised, and it MAY be
    that this is justified, but the blog entry provides no EVIDENCE.

    By the way, savour the irony.  "Erlang's approach [of] using
    separate heaps per process", which he criticises, is in fact
    used elsewhere: ptmalloc does it, the tcmalloc documentation
    makes it absolutely clear that tcmalloc does this also (more
    precisely, it uses a per-thread cache, which is what the "tc"
    part of the name means), and some recent Java systems have
    done the same thing, with a per-thread cache for memory
    management, backed by a shared heap.  The point of the per-
    thread cache is to reduce locking.  There is a spectrum of
    approaches from nothing shared to everything shared, and it
    seems clear that everyone sees merit in not being at either
    extreme.  Progress must be driven by measurement.

(5) He doesn't like HiPE.  For myself, I don't _care_ whether HiPE is
    a JIT or a jackal, as long as it gives me a useful improvement in
    performance.  Inlining across module boundaries _has_ been tried,
    I believe (there's a paper about it somewhere), but it's hard to
    reconcile with hot loading.  HiPE was, of course, a project
    contributed by "the community", and depended on funding which I
    believe has come to an end.  Anyone who wants a better compiler
    should try to find funds for it.  Sun and Google have vastly
    deeper pockets than Kostis Sagonas!

    I think it is particularly unfair to criticise HiPE for having a
    limited range of back ends when it works on *MORE* systems than
    the tcmalloc library he praised.

(6) Erlang is not general purpose.

    But it was never intended to be, and isn't advertised as such.
    
    In fact Erlang loves state, but it wants state to be encapsulated
    within processes.

    "What should you do if you want to deal with a shared-state
    concurrency program in Erlang?"
	Lie down until the feeling passes off.

    If you want shared-state concurrency, I can tell you where to find
    Ada.  I can tell you where to find concurrent ML (Mlton does it OK).
    I can tell you where to find Haskell, which is actually pretty
    amazing these days.

    One can respect Erlang without being married to it.

(7) He doesn't like the syntax.
    Well, it's not quite as much of a disaster as Java syntax, and for
    sure it's not as ugly as CAML or F#.  (They are so ugly that they
    make SML look beautiful, and for someone who prefers Haskell to
    SML on aesthetic grounds, that's saying a lot.)  The funny thing
    is that what makes Erlang syntax clunky is precisely its *similarity*
    to classical languages like Pascal and C...

    Given documentation for BEAM, we might get more alternative syntaxes
    to play with.

    It's fair to point out that Erlang resulted from an experiment with
    several approaches, so responsible steps were taken to make sure that
    it wasn't _too_ bad.

(8) He criticises immutable state on the grounds that while you can share
    tails of a list, you can't share prefixes or infixes.  The answer, of
    course, is multifold:
    - there are immutable data structures where you *can* share infixes
      and you *can* use them in Erlang, it's just that they don't have
      built in syntax.  (For that matter, it would be possible to implement
      Erlang so that slices of tuples could be shared just like slices of
      strings in Java.  SML does this.  In fact, Concurrent SML would answer
      so many of his issues that I'm surprised he didn't mention it.)
    - this is only a problem if you *want* to share prefixes or infixes,
      and somehow I never do
    - in languages with mutable state you cannot safely share ANYTHING.
    The argument that allocating pure objects is a bad match for modern
    hardware is a non sequitur.  Let me quote a paper about Fork/Join
    parallelism for Java: "In many ways, modern GC facilities are perfect
    matches to fork/join frameworks: These programs can generate enormous
    numbers of tasks, nearly all of which quickly turn into garbage after
    they are executed."  That is, generating enormous amounts of garbage
    can be a >good< thing, provided it's the kind that garbage collectors
    manage well.  I've been on the garbage collection mailing list for a
    while, and the Memory Management proceedings have contained papers
    showing that garbage collection can be *worse* for locality (and thus
    modern hardware) and papers showing that it can be *better* for
    locality (and thus modern hardware).  What this means is that one
    cannot simply *assume* "side effects = good for cache, immutability
    = bad", one must *measure*.

(9) He doesn't like the standard library.

    Interestingly enough, I hear the same kind of thing in the Haskell
    mailing list about the Haskell "standard Prelude".  And there is a
    project to develop a new standard Prelude.

    I believe there is general agreement that an improved Erlang library
    could be developed and would be very nice to have.  (I've always
    found the differences between say ETS and DETS more confusing than
    helpful.)

    This is something that can be done piecemeal and by individuals.

    The things I would like to say about the Java libraries would have to
    be displayed on asbestos screens...  Heck, I like the Ada libraries
    less than I used to; the additions are *pointful* but not to my mind
    *tasteful*.

And so it goes.

There *are* things about Erlang that can be improved.
Some things *have* been improved, some things are being improved,
and some things don't need anyone to wait for Ericsson to do them.