[erlang-questions] Rhetorical structure of code: Anyone interested in collaborating?

[Joe Armstrong]

On Thu, Apr 28, 2016 at 5:52 AM, Richard A. O'Keefe <ok@REDACTED> wrote:
> I've been thinking for some time of writing a paper with the
> title "Why can't I see the structure?" based on the idea that
> modules in every programming language I know look like blobs.
> I'm aware of visual notations like UML, BON, SDL, and what
> was it, Visual Erlang?  But for me, those are just spaghetti
> with meatballs; once you get beyond just a handful of boxes
> in your diagram, all diagrams look much the same.  In any
> case, I'm interested in the medium scale.
>
> Why can't I see the structure in a 3000-line module, or even
> a 1000-line module?  (I am not asserting that Erlang is
> particularly bad here.  It isn't.)

[long rambling possibly off topic reply follows]

[you have been warned]


Well I would hope that you don't have to see the structure of such a
module.

To me systems should be made of black boxes that communicate through
well-specified protocols - so the top level description would tell me
how the boxes were connected together and describe the protocols
at a high level of abstraction.

To me the structure is

in state S
     receive
         {X, Pattern1} ->
               ... loads of code I don't need to understand ...
               Y ! Value1

All I should need to know about the module is that it implements the
protocol - How it works should be irrelevant.

This in fact how Erlang was used - in the early days we'd start with
a CCITT spec in SDL - this had the top level - which black boxes there
were, how they were connected and what protocols they obeyed.

Unfortunately the structure (which process starts another process) which
protocol is obeyed is implicit in code and not explicit.

We can only talk about things if they have names and most often we
don't name the protocols used between processes. Saying the
process P1 and modules M1, M2, M3 etc terminate the XYZ protocol
is an extremely precise way of describing what a system does.
Tracing the protocol and comparing to what you expect gives an immediate
way to discover errors.

If I'm debugging code I don't place print statements randomly, but at the
points where messages enter and leave the application - unfortunately
we can't distinguish "important" messages from background chatter.

Erlang was designed to be more-or-less isomorphic to SDL (see
https://en.wikipedia.org/wiki/Specification_and_Description_Language)

Where Erlang was the bottom of three levels of design.

Unfortunately SDL, which was great, got incorporated into UML which was
a *!@#$$*$$$ ing  mess.

Erlang is all about sending messages to things, so message sequence
charts (https://en.wikipedia.org/wiki/Message_sequence_chart) are
brilliant for
describing how Erlang programs work.

When I was at Ericsson we drew MSC diagrams ( the systems guys did this)
then the programmers took these and implemented them - the problem
here is that the MSC diagrams, from which the code was derived,
are not included in the code.

A big problem with code is that we throw away the design documents
necessary to write the code - and don't say what specifications the code
fulfills.

Code that starts with a comment

   %% This code implements the protocol described in [Ref]

Is great


>
> The kind of structure I'm interested in can, I think, be
> described as *rhetorical* structure, like relationships
> between paragraphs.
>
> My *belief* is that if this structure were made explicit,
> perhaps by textual structure, perhaps by annotations, perhaps
> by some graphical form (but probably derived from annotations),
> it would be easier to understand medium-sized wodges of code.

I'm beginning to wonder if the problem is with editors - I use
emacs erlang mode which has no help for structuring and organising
files.

For writing text and lectures I use emacs org-mode with wiki like links.

Hierarchical structure (in org mode) is really easy to see and change
and with wiki links is pretty powerful.

I'm beginning to think that a wiki style of programming might work.

In a wiki if you click on a link and no page exists you are prompted
to write some text - we could make a similar thing to develop
programs.

>
> I'm aware of annotation support in languages like Java and C#
> and for that matter, Smalltalk, but with the exception of
> Smalltalk, nobody seems to be using annotations in this way
> (and that exception is me).
>
> I'd be very interested in hearing from anyone else who has been
> thinking in this area.

Me too :-)

Another idea I had was "monkey patching done right"

The problem is this:

    What I'm writing a module "a" I think
    "I wish module z exported foo, and I wish file.erl exported boo

Now I don't want to have to stop what I'm doing in a and add something
 to z (since then I'll get distracted and mess around with z) - and goodness
changing file.erl is really difficult - I'll have to argue blue in the
face and write
an eep - just think about adding something to lists.erl

In a.erl why can't I say

    -module(a).

    -patch(lists)
     my_super_func(...) ->
    -end.

which adds my_super_func to lists (only it doesn't really) only for me
and not for everybody else.

I think controlling the name space is the way to solve a load of problems
but hierarchical names spaces are a mess, and we can't all agree.

I think we need constructs like

    " what joe call x anne calls y"

I can imaging blocks of code just being named by their MD5 checksums
and having aliases

     alias factorial = "0xad45..c13"  (a md5 checksum)

and naming and discovery mechanism so we could say

    "md5-1" computes the same thing as "md5-2"

The bigger problem of understanding "my" code or "this" code
is " has somebody else already solved this problem".

My feeling is that most problems have already been solved - but
we write code because it's quicker to write code than discover it.

When I get really stuck I post a message to stack overflow and post a tweet.

This is a *fantastic* way to solve problems - but I'm privileged to have
several followers.

Understanding code seems to be related to the idea of making
a minimal kernel that we can understand - so one thing I can't read from code
is "this is the bit you have to understand".

In code reviews I would ask "how does this work" or "what's the most
important stuff" ...

Robert Virding (Blessed be his name) once wrote a very long
module for pattern matching with exactly one comment:

   %% And now for the tricky bit

It was nice to know ...

[I warned you]

Cheers

/Joe


>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions