[erlang-questions] Rhetorical structure of code: Anyone interested in collaborating?

[Joe Armstrong]

On Wed, May 4, 2016 at 2:42 AM, Richard A. O'Keefe <ok@REDACTED> wrote:
> On 3/05/16 7:19 PM, Vlad Dumitrescu wrote:
>>
>>
>> This is true and would be useful, but I think I can see a problem (which
>> might very well be the reason why it isn't something people already do):
>> after a while, there will be literally thousands of discarded stuff. How am
>> I supposed to search and find the alternative that works, especially if I am
>> now working on a different problem than 6 months ago when the stuff was
>> originally written? Is this search faster than starting from scratch? IMHO
>> the answer is that what needs to be saved is a higher-level description, not
>> a complete brain dump of the process.
>
>
> I think Joe Armstrong has already answered this, and with my mention of
> version control
> I thought I had addressed it.  The answer is that we should keep a lot more
> about the
> history of a program than we (specifically including me in that) do, but
> that it does not
> have to go in the source file.
>
> I think we agree that when you are trying to put out fires in the current
> code base,
> you want to read as little as you safely can, and you don't want to be
> distracted
> by stuff that doesn't seem to be relevant.  But when you have decided on a
> fix,
> that's when you need to review the history to see if someone already thought
> of
> it and rejected it for good reason.
>
> I'm reminded of an anecdote I read about the Knuth-Morris-Pratt string
> searching
> algorithm.  (At least, that's what I think it was.)  It got built into a
> particular
> text editor.  A few years later one of the authors was called in and told
> "your
> string search algorithm isn't working.  Fix it."  It turned out that a
> succession
> of programmers had looked at the code, thought it was broken, and "fixed"
> it,
> and all that had to be done to *really* fix it was to put the old code back.
>
> I'm not as good as Joe Armstrong (though I'd like to be).  In some of my
> projects
> I keep a directory dead.d.  Code that seemed like a good idea at the time,
> and may
> still have value as a source of ideas, or at least as an (explained)
> dreadful warning,
> goes there.  For a file foobar.src I often keep a file foobar.txt with
> musings about
> what might or might not go in foobar.src, criticisms of what other people
> have done
> to explain why I am not going to do that, thoughts about refactoring, &c.
>
> I'm not sure that a brain dump of the whole process would be a bad thing,
> as long as you don't *have* to read through it to see the present source
> code.
>>
>>
>> There is also a difference between what is interesting for teaching people
>> (beginners) and what is interesting for future reference for the team that
>> develops the code.
>
> I used to have large chunks of the Discworld stories memorised.
> The story I have in mind at the moment is the contest between Granny
> Weatherwax (who would be the leader of the witches in the Ramtops
> if Discworld witches had leaders, but they don't, so everyone just _knows_)
> and a new witch called, if memory serves me, Diamanda Tockley.
> GW's friend Nanny Ogg is acting as GW's second.
>
> Nanny Ogg [thinks] one day you're going to lose.
> Diamanda: What level are you on?
> Granny Weatherwax: Level one.
> Diamanda [nastily]: Just beginning?
> Granny Weatherwax.  Yes.  Every day.  Just beginning.
> Nanny Ogg [thinks] but it's not going to be today.
>
> Tell it not in Gath, but I am now 60, and have been programming since
> about 16.  Every day, just beginning.  I'm just gone through the Swift
> book and need to do a lot of exercises.  People are telling me I need
> to learn Pony because it's the latest and greatest.  (I've had the Pony
> compiler on my laptop for *months* and still haven't written a program
> with it yet.)  Other people are telling me I need to understand the
> Ambient calculus, and I'm really annoyed with myself because I can see
> that it's *very* closely related to some consulting I did about 10 years
> ago and it would have been really useful to know back then.  Oh, and
> I've been re-learning a language I last used about 1984, and am starting
> to feel fluent in it again.
>
> Yet if you sit me down in front of a body of code, a project, then I am
> a *total* newbie all over again.  With respect to the *programming
> language*, I'm not a beginner.  With respect to the *project*, I really am.
> I've been converting some F# code to Smalltalk (and greatly improving it
> while doing so, if I may say so, boast boast).  And some Python code,
> also to Smalltalk, which Python code turns out to have a *massive* errata
> list on the web, despite having been published by a previously reputable
> publisher) so I'm having to rely much more on the textual description
> than the actual code.  I feel very much like a beginner struggling with new
> stuff, except that this is bigger.
>
> As a specific example, sit me down in front of the Cowboy source code
> and I'm clueless.  NOT because it's bad -- I carefully chose an example
> that isn't -- but because it's a big chunk of code that I am not familiar
> with.
> I'm not a beginner with *Erlang* but I'm a beginner with *Cowboy*.
> I need all the help I can get.

Thank you Richard.

I'm always appalled by people who write no documentation at all and then
say "read the code" as if this were easy. Imagine the PAIN when I read
a comment like this - AAAAAAAAGGGGGGGHHHHH

Reading unfamiliar code is appallingly difficult - you're continually
trying to guess what problem the author was trying to solve.

At best you know what the code does, not what it was supposed to do -
for this you must guess.

Pure functional requirements help bit are not the full story -
it's the "subject to the requirements that ..." which is the killer.

That what? - "that it's maintainable, efficient, small, fault-tolerant,
understandable, ...

I've done code reviews - virtually every time I've red penned some code
and said why is it done this way, there turns out to be a non-obvious
and undocumented reason why.

I'm reminded of a visit to Aachen (I think) - we were show a room where
a hermit had been walled in for 20 odd years, getting his food through
a hole in the wall.

The Guide said " nobody knows why he did this, but there is
a reason for everything you just have to ask why"

This is like reading code - it's looks weird and there must be a reason
but you don't know why, and you can't ask a text file a question.

I'd be happy if could understand my own code a few years after I've written
it - and *overjoyed* if I could understand other peoples code.

Often the effort of "reimplementation from scratch" is less than that
of understanding and changing legacy code.

Case in point: Once I had to modify some open LDAP code written in
C - the code was appalling difficult to understand (I know, it's because
I'm the world's worse C programmer) - but nevertheless..

We didn't need all of LDAP, we were using one or two hard coded
lookups - I gave up and wrote custom code in Erlang over the lunch break.

- I have said several times that we write code because it is quicker to
write it from scratch, than to discover code that does what we want, or
modify code that does almost what we want but not quite.



This is crazy - but as yet I know of no better method and I've been
searching for a better method for as long as I've been programming.


Using libraries is a double edged sword - great if they work -
horrible if they need a slight modification because they do almost
what you want but not exactly.

I have wasted many hours *assuming* a library was correct, and my
code was bad - only to find the opposite - Grrrrrrrrrrrrr

This is particularly bad when the library/framework is highly rated and
supposedly well-tested - so when I get an error I assume I've made a mistake
I start looking at my code, not the library.

Then when the evidence become overwhelming, you start reading the
library code, and Ouchhhhhh - you have to understand it - which may or
may not be easy.

Programming is a craft - not a science or art.

It should be a science but we're not there yet.

/Joe



>>
>> These categories need different levels of detail, different kind of
>> information.
>
> (a) I am not sure that the categories are all that distinct, and
> (b) absolutely.  (I'm reminded of the tracing facilities in Arthur Norman's
> symbolic integration program, my first introduction to serious logging.)
>
> Adequate documentation tools let you annotate a chunk of text with
> the level it's appropriate for (like the black or white page corners in
> Wheeler Thorne and Misner's "Gravity", marking out basic and advanced
> sections, where it looks like I'm *never* going to cope with the advanced
> sections).  Good documentation tools make it easy for you to view a
> document with only the material appropriate to you visible.
>
> One more anecdote.  When I joined Quintus, I was handed a ring binder
> and told "this is the internal documentation.  It will help you find your
> way around the code."  I promptly got lost and couldn't find a
> certain file.  "Oh, we forgot to tell you.  It's out of date.  Half of it is
> wrong.  We didn't bother keeping it up to date because we knew that
> stuff anyway."
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions