[erlang-questions] Rhetorical structure of code: Anyone interested in collaborating?

[Lyn Headley]

I recently conceived a sociotechnical process that would help us
understand each other's code. It is simple to implement. In bare bones
it is this:

I am new to your project. As I read your code, certain events are
recorded by my editor and sent to a log that you are monitoring. You
can easily browse the log, see where I am bumping into issues, and
easily provide feedback.

That's it.

To put some flesh on it, you might see:

Lyn jumped from file:line to function f/2 in file2.
Lyn expressed puzzlement.

etc.

Your feedback could be in the context of this series of events as
well. So when I receive it, I am reminded of exactly where I was. As
feedback I see the annotated log:

Lyn jumped from file:line to function f/2 in file2.
Lyn expressed puzzlement.
[Joe said: That's an optimization for X case].

That's really it.  This is easy to write and I think it's a huge win.
What do you think?





On Wed, May 4, 2016 at 7:52 AM, Joe Armstrong <erlang@REDACTED> wrote:
> On Wed, May 4, 2016 at 2:42 AM, Richard A. O'Keefe <ok@REDACTED> wrote:
>> On 3/05/16 7:19 PM, Vlad Dumitrescu wrote:
>>>
>>>
>>> This is true and would be useful, but I think I can see a problem (which
>>> might very well be the reason why it isn't something people already do):
>>> after a while, there will be literally thousands of discarded stuff. How am
>>> I supposed to search and find the alternative that works, especially if I am
>>> now working on a different problem than 6 months ago when the stuff was
>>> originally written? Is this search faster than starting from scratch? IMHO
>>> the answer is that what needs to be saved is a higher-level description, not
>>> a complete brain dump of the process.
>>
>>
>> I think Joe Armstrong has already answered this, and with my mention of
>> version control
>> I thought I had addressed it.  The answer is that we should keep a lot more
>> about the
>> history of a program than we (specifically including me in that) do, but
>> that it does not
>> have to go in the source file.
>>
>> I think we agree that when you are trying to put out fires in the current
>> code base,
>> you want to read as little as you safely can, and you don't want to be
>> distracted
>> by stuff that doesn't seem to be relevant.  But when you have decided on a
>> fix,
>> that's when you need to review the history to see if someone already thought
>> of
>> it and rejected it for good reason.
>>
>> I'm reminded of an anecdote I read about the Knuth-Morris-Pratt string
>> searching
>> algorithm.  (At least, that's what I think it was.)  It got built into a
>> particular
>> text editor.  A few years later one of the authors was called in and told
>> "your
>> string search algorithm isn't working.  Fix it."  It turned out that a
>> succession
>> of programmers had looked at the code, thought it was broken, and "fixed"
>> it,
>> and all that had to be done to *really* fix it was to put the old code back.
>>
>> I'm not as good as Joe Armstrong (though I'd like to be).  In some of my
>> projects
>> I keep a directory dead.d.  Code that seemed like a good idea at the time,
>> and may
>> still have value as a source of ideas, or at least as an (explained)
>> dreadful warning,
>> goes there.  For a file foobar.src I often keep a file foobar.txt with
>> musings about
>> what might or might not go in foobar.src, criticisms of what other people
>> have done
>> to explain why I am not going to do that, thoughts about refactoring, &c.
>>
>> I'm not sure that a brain dump of the whole process would be a bad thing,
>> as long as you don't *have* to read through it to see the present source
>> code.
>>>
>>>
>>> There is also a difference between what is interesting for teaching people
>>> (beginners) and what is interesting for future reference for the team that
>>> develops the code.
>>
>> I used to have large chunks of the Discworld stories memorised.
>> The story I have in mind at the moment is the contest between Granny
>> Weatherwax (who would be the leader of the witches in the Ramtops
>> if Discworld witches had leaders, but they don't, so everyone just _knows_)
>> and a new witch called, if memory serves me, Diamanda Tockley.
>> GW's friend Nanny Ogg is acting as GW's second.
>>
>> Nanny Ogg [thinks] one day you're going to lose.
>> Diamanda: What level are you on?
>> Granny Weatherwax: Level one.
>> Diamanda [nastily]: Just beginning?
>> Granny Weatherwax.  Yes.  Every day.  Just beginning.
>> Nanny Ogg [thinks] but it's not going to be today.
>>
>> Tell it not in Gath, but I am now 60, and have been programming since
>> about 16.  Every day, just beginning.  I'm just gone through the Swift
>> book and need to do a lot of exercises.  People are telling me I need
>> to learn Pony because it's the latest and greatest.  (I've had the Pony
>> compiler on my laptop for *months* and still haven't written a program
>> with it yet.)  Other people are telling me I need to understand the
>> Ambient calculus, and I'm really annoyed with myself because I can see
>> that it's *very* closely related to some consulting I did about 10 years
>> ago and it would have been really useful to know back then.  Oh, and
>> I've been re-learning a language I last used about 1984, and am starting
>> to feel fluent in it again.
>>
>> Yet if you sit me down in front of a body of code, a project, then I am
>> a *total* newbie all over again.  With respect to the *programming
>> language*, I'm not a beginner.  With respect to the *project*, I really am.
>> I've been converting some F# code to Smalltalk (and greatly improving it
>> while doing so, if I may say so, boast boast).  And some Python code,
>> also to Smalltalk, which Python code turns out to have a *massive* errata
>> list on the web, despite having been published by a previously reputable
>> publisher) so I'm having to rely much more on the textual description
>> than the actual code.  I feel very much like a beginner struggling with new
>> stuff, except that this is bigger.
>>
>> As a specific example, sit me down in front of the Cowboy source code
>> and I'm clueless.  NOT because it's bad -- I carefully chose an example
>> that isn't -- but because it's a big chunk of code that I am not familiar
>> with.
>> I'm not a beginner with *Erlang* but I'm a beginner with *Cowboy*.
>> I need all the help I can get.
>
> Thank you Richard.
>
> I'm always appalled by people who write no documentation at all and then
> say "read the code" as if this were easy. Imagine the PAIN when I read
> a comment like this - AAAAAAAAGGGGGGGHHHHH
>
> Reading unfamiliar code is appallingly difficult - you're continually
> trying to guess what problem the author was trying to solve.
>
> At best you know what the code does, not what it was supposed to do -
> for this you must guess.
>
> Pure functional requirements help bit are not the full story -
> it's the "subject to the requirements that ..." which is the killer.
>
> That what? - "that it's maintainable, efficient, small, fault-tolerant,
> understandable, ...
>
> I've done code reviews - virtually every time I've red penned some code
> and said why is it done this way, there turns out to be a non-obvious
> and undocumented reason why.
>
> I'm reminded of a visit to Aachen (I think) - we were show a room where
> a hermit had been walled in for 20 odd years, getting his food through
> a hole in the wall.
>
> The Guide said " nobody knows why he did this, but there is
> a reason for everything you just have to ask why"
>
> This is like reading code - it's looks weird and there must be a reason
> but you don't know why, and you can't ask a text file a question.
>
> I'd be happy if could understand my own code a few years after I've written
> it - and *overjoyed* if I could understand other peoples code.
>
> Often the effort of "reimplementation from scratch" is less than that
> of understanding and changing legacy code.
>
> Case in point: Once I had to modify some open LDAP code written in
> C - the code was appalling difficult to understand (I know, it's because
> I'm the world's worse C programmer) - but nevertheless..
>
> We didn't need all of LDAP, we were using one or two hard coded
> lookups - I gave up and wrote custom code in Erlang over the lunch break.
>
> - I have said several times that we write code because it is quicker to
> write it from scratch, than to discover code that does what we want, or
> modify code that does almost what we want but not quite.
>
>
>
> This is crazy - but as yet I know of no better method and I've been
> searching for a better method for as long as I've been programming.
>
>
> Using libraries is a double edged sword - great if they work -
> horrible if they need a slight modification because they do almost
> what you want but not exactly.
>
> I have wasted many hours *assuming* a library was correct, and my
> code was bad - only to find the opposite - Grrrrrrrrrrrrr
>
> This is particularly bad when the library/framework is highly rated and
> supposedly well-tested - so when I get an error I assume I've made a mistake
> I start looking at my code, not the library.
>
> Then when the evidence become overwhelming, you start reading the
> library code, and Ouchhhhhh - you have to understand it - which may or
> may not be easy.
>
> Programming is a craft - not a science or art.
>
> It should be a science but we're not there yet.
>
> /Joe
>
>
>
>>>
>>> These categories need different levels of detail, different kind of
>>> information.
>>
>> (a) I am not sure that the categories are all that distinct, and
>> (b) absolutely.  (I'm reminded of the tracing facilities in Arthur Norman's
>> symbolic integration program, my first introduction to serious logging.)
>>
>> Adequate documentation tools let you annotate a chunk of text with
>> the level it's appropriate for (like the black or white page corners in
>> Wheeler Thorne and Misner's "Gravity", marking out basic and advanced
>> sections, where it looks like I'm *never* going to cope with the advanced
>> sections).  Good documentation tools make it easy for you to view a
>> document with only the material appropriate to you visible.
>>
>> One more anecdote.  When I joined Quintus, I was handed a ring binder
>> and told "this is the internal documentation.  It will help you find your
>> way around the code."  I promptly got lost and couldn't find a
>> certain file.  "Oh, we forgot to tell you.  It's out of date.  Half of it is
>> wrong.  We didn't bother keeping it up to date because we knew that
>> stuff anyway."
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions