[erlang-questions] Is there a good source for documentation on BEAM?

[Richard O'Keefe]

On 29/05/2012, at 3:25 AM, Thomas Lindgren wrote:
> BEAM docs are not needed to produce a second source implementation, as shown by several examples. (1) 

People were not asking for BEAM documentation in order to produce
a second source implementation.  That's a red herring.  People
ask for BEAM documentation in order to understand and perhaps
revise the main implementation.

One thing that Open Source is about is about community contributions
and it is awfully hard to contribute to something that is
under-documented.
> 
> Also, there has so far been little practical interest shown by Erlang users in such a second source. So implementation efforts may be in vain. (3)

Again, the red herring.  This is *NOT* about second implementations.
All of the additional implementations I've ever heard of used their
own back end, *not* BEAM, and would *not* have received any great
benefit from BEAM being documented.

There is detailed documentation available for Lua and Icon and the
WAM and several other VMs around, so there's not even any great
advantage in learning about VMs from BEAM.

> My personal view, at least, is that most of the difficulty in "keeping up with The Real Thing", Erlang/OTP, would be not in reproducing BEAM but in writing a fully compatible implementation tracking the rest of the runtime, ERTS. (2)

Well, not _just_ that, but that's a big part of it.

> So, is there a _practical_ case for doing these docs? In particular, will the effort result in useful external contributions that outweigh time spent? Not at all clear to me.

Bearing in mind that "BEAM documentation" and "second system implementation"
are about as irrelevant to each other as any two topics about the same
language can be, what _might_ we realistically expect from BEAM documentation?

(1) There are now several languages implemented on top of Erlang.
    Some are interpreted in Erlang, some are compiled to Erlang ASTs,
    and some are compiled to BEAM.  Compiling to the BEAM is *much*
    harder than it needs to be because of the current undocumentation.

    This isn't *replacing* Erlang/OTP but *augmenting* it.

(2) I've proposed a number of minor but handy extensions to Erlang
    syntax that lack a reference implementation because I have no idea
    what the compiler should generate for them; these could use the
    existing BEAM.

(3) There are existing things that could be compiled more efficiently.
    I'm thinking here in particular of list comprehension.  I'm not
    sure if the improved translation can be done with the existing
    BEAM or if minor extensions would be needed, because the
    undocumentation for the BEAM does not make clear any range
    limits or other restrictions on BEAM instructions.

(4) There are even bigger changes, like frames, where even estimating
    the scope of the changes is hard because of the undocumentation.

But above all you are making an assumption which I utterly reject,
namely that documentation is a COST and ONLY a cost, that producing
documentation provides no DIRECT benefits to the people writing the
documentation.

On the contrary,
 - you may find defects as you document
 - you may be able to structure the documentation so that
   test cases can automatically be extracted
 - you may in the very act of explaining a limitation see
   how it can be overcome
 - you may be able to extract parts of the implementation
   automatically from the documentation
and I could go on.

There's a slogan I learned from a business textbook:
  find the indispensable man and fire him!
There was something I found unsettling: we were told in this 
thread that there wasn't any need for documentation because
the Erlang/OTP maintainers had enough people who _knew_ this
stuff.

One of my colleagues here was managing a software project
once; a key employee went on holiday and was murdered in a
far country.  A former student of mine was starting up a
new company with me as a consultant, and being a worse
driver than he thought, drove at speed into a tree.  The
tree survived.  One of my former colleagues, a very
intelligent and likable guy, was cycling down a Melbourne
street and got knocked over by a hit-and-run driver.  The
result was head injury and someone who could dress himself
but couldn't program if his life depended on it.  When I
was at Quintus, the founder who wrote the compiler and was
the only person who really understood it quit and was never
heard from again.

Fergus O'Brien supervised an MSc on "Organisational
Forgetting" and the fact of organisational forgetting has
haunted the fringes of my mind ever since.

People die.  People get head injuries.  People quit.
Documents get lost.  (If you ever find an architecture-
of-the-ICL-2900 manual, I'd like to see it.)  Even if
things _are_ written down, organisations forget _where_.
If they aren't written down, they WILL be forgotten
sooner or later.


People 
Well, my colleague Andrew Trotman once had a key employee




We
> 
> Hope it helps.
> 
> 
> Best regards,
> Thomas
>