[erlang-questions] FOP (was: Re: Trace-Driven Development)

[Richard O'Keefe]

On 8/06/2012, at 7:19 PM, Michael Turner wrote:
>>>> Our first technical writer, Kennita Watson, was technically competent.
>> 
>> Did you read that sentence?
> 
> Yes. I read it as "technically competent *as an engineer*", because
> 
> (a) she has degrees in computer science (from Stanford and MIT at that
> -- facts I recalled about her the instant I read her name), and
> 
> (b) what followed was an anecdote that I didn't quite get, because
> somebody (you?) wrote "Never let an engineer write the documentation"
> and I thought you were agreeing with the point -- you certainly didn't
> contradict it immediately.

She wasn't an engineer >>>ON OUR PROJECT<<<.
And that's what I was agreeing with.
> 
> So at first I read your comment on her work as saying that she'd
> suggested changes in the documentation to make things more
> documentable from *her* point of view (as an engineer), but that they
> were still things that the customers didn't understand.

Not a bit of it.  Our customers were *also* programmers; her value was
precisely that her perspective was their perspective.  I did say it was
Prolog; what other kind of person uses a Prolog system?

> Such people are admittedly as rare as hen's teeth in the more
> quotidian livelihood of tech writing. The Kennitas of that world are
> rare enough. (I think she was only sidelined because of some injury --
> I'm not sure. I'd kill to make use of her on a project myself, if she
> were still available.)

"Sidelined"?  She moved to a bigger company (which is still around,
unlike Quintus) with a larger customer base and more to write about.
I never asked, but they could certainly have _afforded_ to pay her
more than we could.  Everyone I was close to at Quintus wanted her to
stay.
> 
> Which takes me back to my agenda: what's the *practical*, *dirt-cheap*
> way to *eventually* get better quality docs for Erlang/OTP?

Step 1:  fix the tool chain so I can build the documents.

I've annotated the erl_docgen document; there's something that needs
fixing on every page.  I am keen to send in patches, but I can't do
that until I can build the documents.

> And their edits can easily be reviewed.

And now we come to the crux of the matter.  It's the *review* process.
That takes time and effort and management, and it's not at all clear
to me what, if anything, makes a wiki easier to manage than the current
setup.  The mere existence of a wiki is no guarantee of good reviewing.

The great thing about the current scheme is that edits are reviewed
BEFORE they are adopted, but we're promised that they'll be reviewed
quickly.  With a Wiki, bad stuff goes in, and if you're lucky it will
be reviewed and eventually be taken out, but there _is_ that window
where the bad stuff is there.

I won't mentioned which Wikipedia page it was, but the one Wikipedia
page I ever did anything to was one which contained several gross
factual errors and had for months.

But there is also the fact that wikis tend to lead to lots of bits
all over the place and a hell of clicketyclicketyclickety for almost
anything.  Recently I've been wandering round the Tizen "documentation".
Not a good advertisement for "Web documentation".

One thing the current tool chain gets almost right is that it really
does generate man pages and HTML and PDF from the same sources.  I
have a shelf of printed and bound Erlang manuals.  

> People will say a lot of silly things about wikis and their governance
> until they gain more direct experience of maintaining them. In that,
> wikis are like anything in life.

My comments about wikis are based entirely on my experience of actually
trying to get basic information out of wiki "documentation" for several
projects.  "Like anything in life", wikis obey Sturegon's law.

>> Wikis do not solve this problem, they exacerbate it.
> 
> You're prone to sweeping generalizations about wikis based on no
> direct personal experience of editing them, from what I can tell.

I have edited one page in one wiki, no wait, three pages in two.

But editing is simply not the issue.
My comments are based on too much painful experience of trying
to *USE* wikis edited by others.
I don't see why only people who edit wikis count here.
I think that people who *USE* documentation matter, and that
having tried to *use* about a dozen wiki-"documented" open source
projects and finding intense frustration is relevant.

I find the suggestion that only experienced editors of Wikis are
competent to judge the usability of wikis outrageous.

>> How high *is* the bar?  Have you ever submitted a documentation patch
>> and had it rejected?
> 
> I've never submitted a documentation patch.

Ah!  You've never tried, but you just _know_ that the barrier is high.

> You hate amateurs.

No I do not.  I *am* an amateur.

Let's go back to Quintus.  While we had technical writers,
the "engineers" often *originated* documentation.  It just had to
go through technical writers before customers saw it.  And it was
the same for code:  with one notable exception, all changes that
affected what customers saw had to be reviewed before final
acceptance.  And that one exception was the single dumbest thing
Quintus ever did, and sadly, one of the most copied.

The chief problem with the existing Erlang documentation is that
it was released without an adequate review of the language.

> I can understand that. But the experts fail sometimes too.

It's not professional vs amateur.  It is review-before-release
vs post-release-review-if-ever.

By the way, if you want to see an appalling example of editing,
take a look at the ANSI Smalltalk standard.  The 5th draft was
full of glitches, almost all of which survived in the published
standard.  Even being-published-by-ANSI was no guarantee of
adequate proof-reading.
> 
> If I submitted a patch to the seq_trace documentation so that it
> mentioned that seq_trace implements Lamport clocks, and it got
> rejected with "we're still thinking about how to work that fact in,
> maybe we'll have something in the next release," I think I'd end up
> writing Lamport to say, "Can you *believe* these people?"

Ulf Wiger is not in charge of reviewing changes to the Erlang
documentation.  The people who _are_ in charge might well be
perfectly happy with adding "Since its first release, seq_trace
has relied on Lamport clocks [ref], but it is not guaranteed
that it will always do so."

Right now, I'm reading someone complaining about the extreme
difficulty of putting in a sentence and making up a story about
what he'll say to someone when rejected

	WHO HAS NOT TRIED IT.

Try it.  *Then* resume this conversation.

No, wait.  *I've* just posted a change to erlang-patches.

> Good ideas, I'm sure. I'm using Erlang to implement a computational
> linguistics framework, so I'm definitely keen on those sorts of
> solutions myself. But automating a task to get "better than nothing"
> shouldn't be the goal here. This is supposed to be documentation by
> professionals, for professionals.

And professionals use good tools.  INEX isn't running "link-the-Wiki"
any more because you need to able to compare what you can do with a
"gold standard", and it turns out that while the Wikipedia is
*copiously* linked, it isn't always *well* linked, and when programs
did badly at matching what was there, it was sometimes because the
programs were doing better.

As for unimplemented EEPs, that comes back to a documentation issue,
namely that the BEAM is not (yet) adequately documented.