[erlang-questions] Erlang and bbcode
Fri Jul 13 14:11:18 CEST 2012
On 13 Jul 2012, at 13:04, Joe Armstrong wrote:
> Since they have no specification, their can't really be any tests.
> Or worse, in the absence of a specification, the tests *become* the
Well, in general, I'd have to take exception to this description ;-),
but in the case of a markup language, pushing through without
a specification is obviously asking for trouble.
With common commercial development, there is very
seldom a specification in any useful sense of the word. What
there is, is a customer's more or less vague idea of what they
want, and the reality any implementation of their idea will
In such situations, the tests *do* become the specification,
because a specification *cannot* be written! If you try to write
one anyway, it will seriously delay the project, and be obsolete
as soon as it's finished, if it ever had a chance of being correct
(it probably didn't).
In other words, don't blame the designer who creates a test
suite in those cases, rather than brow-beating the poor customer
until they come up with a workable specification. The practice
of using cue cards to write user stories (and the acceptance
criteria on the back) evolved very much because it was a
workable way of discussing the requirements with users,
whether or not they had any engineering or science background.
It seems ridiculously vague and limited, but in practice, it actually
helps to catch *some* of the information needed to arrive at some
mutually understood acceptance criteria. This wasn't created
because people didn't understand what formal specifications,
or thorough testing, looked like - it was created in order to achieve
at least *some* level of improvement with *real* projects and
Besides, when did you last see a good specification of how
a Web GUI should behave - or a good way to test it? *
Another example: When I gave an invited talk at NWPT'99  in
Uppsala, Sweden, a member of the audience (probably a PdD
student), reacted to my description of the 3-month test period
before release of the AXD 301, saying "That's ridiculous! If you
had written it using a language that could be formally verified,
you wouldn't have to test at all!".
I mumbled something about not knowing if that would have been
at all possible, and Thomas Arts jumped in and offered to argue
the point with him in the break. It was a very vivid discussion, as I recall.
Obviously at that scale (for all except for that PhD student), there
is no chance of arriving at a coherent formal specification, since
the standards we were supposed to comply with were not
themselves formally specified**, and many of the customers didn't
necessarily want *exactly* what was in the standard anyway.
Furthermore, many of the non-functional requirements were not
specified at all, except as a more or less vague list of "goodness
criteria", trying to encapsulate what it means for a system to be
What we ended up with was a large pile of informal requirements,
and a huge set of test cases (nearly a million lines of code).
At that point, the test suites *were* indeed the real specification,
albeit an extremely bulky and inaccessible one.
* One of the best is probably Cucumber (http://cukes.info/), which
is based on cue cards, and relies on quite an ambitious mapping
to the underlying web stack (e.g. Rails). Without such a mapping,
it is less useful, although it still has its sweet spots.
** …although some would disagree. They are certainly specified in
a bureaucratic way, but that's not the same thing.
Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc.
More information about the erlang-questions