[erlang-questions] yet more ranting about Erlang docs

[Jayson Vantuyl]

>> Side-rant: Why isn't there a search box on EVERY PAGE?
> 
> Because the code that generated the web site didn't add a search
> box.
You are clearly not an interface architect.  Considering that this is a serious impediment to learning Erlang (and even using it as we collect more scattered functionality), it should be a priority, and it appears to be one on the erldocs site.

The subsequent text seems to indicate that there is limited time and resources to improve docs.  That's actually a pretty acceptable answer.

> Marketing airhead - no. The only marketing people just tried to sell
> Erlang - they had no influence on the terminology.
No marketechture.  Good thing, that.

> "Application" was just a name for a "directory with stuff in it" that
> obeyed certain principles. Things inside an application could be
> started and stopped, code in an application could be changed and so
> on. Most applications were active (things like databases, web servers
> etc) - some were pure code (stdlib) - the pure code directories (about
> 5% of the total) were anomalies - so we shoe-horned them into
> applications - to make everything consistent. A bit silly because
> you can't stop and start a library, but you can code_change it :-)
The unhelpful thing about terms like "application" and "server" is that everybody seems to think that they are imbued with certain purposes.  In this respect, I don't envy you, ask most of this terminology wasn't pinned down when Erlang started using them anyway.

Just for clarification, I think the above indicates that "Applications" are a packaging and deployment concept, not a program architecture concept.  In other words, at deployment, you deal in these applications; but at runtime, those applications aren't visible.

This is confusing in that the term has been co-opted to by the application framework (i.e. the OTP application module), which uses the same name, but has an entirely different purpose (i.e. managing components at runtime).  There is some correspondence, but not enough.

As if that weren't bad enough, the term has been co-opted AGAIN for the grouping of documentation.  This is problematic because the term is now used in three different ways, with each one not being an exact mapping of any of the others.

This is extra confusing because, in tiny programs, you actually can have one "deployment" application, one "runtime" application, and one "documentation" application--each with the same name.  The metadata for them is also mingled together, even though the individual tools are largely agnostic to the bits they don't use.  This makes for some nonintuitive behaviors and a confusing workflow.  Of course, it's only when you grow a project (to the size of OTP, for example) that the holes in the conceptualization start to show.

"Applications" at runtime (i.e. what you see in appmon), everything seem to be fine.  This appears to be the original concept, and seems to be well-factored.  Still, it's pretty clear that Erlang's documentation and deployment systems need to be revamped to make this less confusing.

Unfortunately, the deployment stuff seems to have critical mass outside of Ericsson, with things like rebar.  These projects don't have much inclination to support the existing deployment stuff, so it is unlikely that Ericsson will ever adopt their work.

The center of mass for documentation is still at Ericsson.  Unfortunately, without a real concern for fixing the overall structure of the documentation, I don't expect to see too much for a while.  I don't think that books will help in that respect.  They will only make the information necessary to use Erlang more confusing, decentralized, and contradictory.  Understanding a language is usually something to do before investing in a book.

There's just no excuse for keeping the barrier to entry artificially high.  It might seem laughably obvious that someone who isn't willing to pay the money for a good Erlang book has nothing to contribute.  This is not true.  They taught me Java in college, and I learned Python because the docs were free on the Internet.  Guess which one has served me better?

Or, more pessimistically, look at the horror that is PHP.  Do you think it's been successful because of the books?  And don't rely on arguments of technological superiority.  If Facebook can be built largely on PHP, that's enough to indicate that we should be trying harder to compete.

> No idea - personally I find most language websites pretty
> incomprehensible - I have never found leaning any new language
> a pain-free process - I have spent most of my working life staring
> at strange error messages wondering what they mean.
> 
> If you're put off by strange terms, lousy documentation, incorrect
> examples, code that doesn't work, explanations that are plain wrong
> and idiotic specifications you shouldn't be a programmer.
Wrong.  If you're put off by these things, perhaps you shouldn't be a language designer, but there is absolutely no reason that things need to stay difficult.  Not every good programmer should have to climb the mountain "because it's there".  Putting in a tunnel has a number of benefits, as we can all get started on different mountains that have never been climbed before.

> In my book a "smart programmer" is one who despite these minor
> obstacles can figure out how things work.
I can figure this out.  It wastes a lot of my time doing it.

> Would you hire a "smart programmer" who rejected a language
> because the documentation was crap?
I'm still here, right?  Look, smart programmers don't need to spend all of their days wrestling with minutia and mundane tasks because their tools are too ill documented and obtuse (which prevents easily developing better tools as well).  Smart Programmers don't reject a language because the documentation is crap.  They might do so because the people behind the language refuse to make it better.  Don't be that guy.  See Wheaton's Law.

> One sign of a smart programmer is that they recognize that the
> documentation *is* crap.
Sure.  And yet they often go to the documentation first, rather than the source.  This is because sometimes you need example code, lists of expected return values, etc. without having to pore through fourteen pages of code.  This whole thing got started because of nuances of guards.  In this situation, the documentation was less than stellar, and they probably would have had to dig through C to figure this one out.  This is not a winning strategy.

> If I found a programmer who said "this documentation is great"
> I would be really suspicious - don't hire this guy :-)
Agreed there.

> Sounds like you're a smart programmer, having found the odd
> inconsequential wart in our otherwise incomparably magnificent documentation.
I applaud your humor, but I'm deadly serious about this.

While limited resources are perhaps the best reason for the documentation not being better, please do not trivialize the importance of documentation.  Smart Programmers might be identifiable by the ability to succeed despite the documentation.  That does not mean that we should deprive them of the unenviable task of being supported by junior programmers (who will *always* read *only* the docs).  Perhaps if you had a few of those to work with, they could provide the resources necessary to do things like improve the documentation.  I bet they'd do it cheaper than smart programmers.  Better yet, they might build their resume until they accidentally become Smart Programmers by osmosis.

I'm building businesses on Erlang.  My house payment and the food on my family's table is based on this decision.  I did it because Erlang has promise, gets a number of things right, and is reasonably open.  I did it because Erlang makes some important things easier than Java, Ruby, or Python.  I do not want to have to spend my days doing everything by myself because Erlang has a "learning cliff" instead of a "learning curve".  I want to be able to scale my programming teams, as well as my programs.

Since I'm not an Ericsson employee, I (perhaps ironically) have to be a little more vigilant about shepherding that investment.  Please, don't try to justify crap, and don't try to classify anyone who wants it to be better as some sort of mediocre programmer.  Just say:

* It takes time and money.
* Our customers mission critical needs are always our top priority.
* When we can make it better, we want to and we will.
* We accept patches.

Better yet, see if you can find someone in the community that has an idea for a way to organize the docs that makes sense, and then figure out how to roll the information in the old docs into the new ones.  Heck, if you want to be really crazy, try re-imagining the structure of your deployment "applications", documentation "applications", and runtime "applications" into something coherent.  I think that this is where we need to go, but if we don't know where we're going, we'll never get there.

-- 
Jayson Vantuyl
kagato@REDACTED