[erlang-questions] Trace-Driven Development

[Michael Turner]

On Wed, Jun 6, 2012 at 9:40 AM, Henning Diedrich <hd2010@REDACTED> wrote:
> I think from the pyre, this must be salvaged as the actual point I was going
> for:
>
>
> On 6/4/12 6:52 AM, Michael Turner wrote:
>
>> I'm now
>> doing unit testing on a module as I develop it further, based on
>> collecting and filtering seq_trace results
>
>
> Is this a common approach, who else does this and how?

I'd like to know. I'm guessing that one of the big problems I've had
with understanding Ulf here is that, for him, tracing is, by
definition, a way to collect data about *anomalous* behavior. To me,
tracing is "selectively (and only on occasion) collecting data about
behavior." Period. You can do whatever you want with that data. The
behavior doesn't have to be pathological. In fact, you can use the
data as some assurance of correctness - the "occasion" can be running
a test suite. Which is to say, the "trace-driven development" of this
thread.

Is it problematic? Of course it is. I accept the possibility that some
of my seq_trace-enabled tests could pass with flying colors only
because the delays incurred by recording the trace tokens suppress
some race condition that actually does arise in production runs. But,
hey, that's testing for you: it can only prove the presence of errors,
not their absence.

On your point about documentation (which is basically a good one,
don't get me wrong):

Jason:
> "Erlang tracing is a seething pile of pain that involves reasonably
> complex knowledge of clever ports, tracing return formats, and
> specialized tracing MatchSpecs (which are really their own special
> kind of hell). The tracing mechanism is very powerful indeed, but it
> can be hard to grasp."

Me:
> Obviously, that kind of statement has no place in the official
> documentation of a professional product.

(Which Ulf agrees with!)

You write:
> Honest statements like the one you are citing may be the actual luxury and
> strong value of Open Source efforts, which are not directly a commercial
> product. Possible only because they /are/ not a product. For one user, fresh
> air like this only creates trust and allows to (even very precisely) set
> expectations and alertness to shortfalls in both documentation and package.
>
> That is not to paint the situation rosy, it would be lovely if it was
> better. But I appreciate the honesty, very much.

There are three issues here:
(1) how honest one should be in a given context,
(2) in what style one should couch that honesty,
(3) whether Erlang is "directly" a commercial product.

Last one first: we hear a lot in this forum about how Erlang/OTP falls
short in one way or another because of "priorities". These priorities
are those of a company, Ericsson, that has ways of making money off of
Erlang/OTP. Erlang/OTP is thus an open source effort *and* a
commercial product. You can say it's not "directly" commercial,
because they don't sell Erlang/OTP distros. But an economist would
say, "It's just the degenerate case of the shelf price being zero."
There are plenty of software products where the non-zero shelf-price
is negligible compared to all the other costs to the user -- support
contracts, buying add-ins, employing consultants to configure the
thing, and to reconfigure it, and so on.

Being a product means Erlang/OTP gets the best of both worlds in some
ways, but also the worst of both worlds in other ways. This thread
dissolved into acrimony between me and Ulf, and I believe largely
because seq_trace has been somewhat the victim of a "worst of both
worlds" scenario, of a kind I can only speculate about since the
disambiguating details are behind the corporate veil.

How that relates to the first two issues:

I appreciated Jason's wording when it was in the context of his blog.
Unfortunately, when those words are put in the documentation for a
corporation's product, they convey the message "Wow, we got some heavy
shit here -- but you need it, so you better buy one of our training
courses run either by us or some of our alumni, to help get you
through it." I.e., it comes across as neither appropriate in style nor
exactly ethical in the promotion of services. (Pretty bad advertising
either way.)

On a blog, I can take Jason's"special kind of hell" with a grain of
salt, and with real amusement. But on what amounts to a corporate
website? As long as we're extolling honesty here, let's say it: isn't
Ericsson paying the bills for erlang.org? Passages like these, in
*that* context, leave me feeling confused and disturbed. (But they
also diverted me to seq_trace, which it turns out is quite the nugget
of gold. So it's not all bad.)

While we're on appropriate style, let me point out the kind of thing I
prefer. Steve Johnson, in his yacc paper, wrote this:

"... it is better that the keywords be reserved; that is, be forbidden
for use as variable names. There are powerful stylistic reasons for
preferring this, anyway."

and, in a version long since gone, he added a footnote:

  * He says, weakly.

It made me sad to see that footnote disappear, as AT&T Unix distros
became ever more official in the early 80s. The footnote had
epitomized his quietly self-deprecating wit. Removing it left him
sounding like some kind of pontificator he never was (at least as far
as I could tell from his talks at U.C. Berkeley back then, which were
always standing-room-only). Official documentation has to be official.
But it doesn't have to be overbearing. The best stuff always reminds
you, every few pages, that there's a human being back there somewhere,
who only wants to help.

By all means, write vividly and well. Especially on blogs. But if
there's a blog entry that describes an aspect of your product as  "a
special kind of hell," that's a signal that you need to improve your
documentation (and maybe your product) not simply echo the sentiment.
Oh, yes, I know: "priorities." *Sigh*.

-michael turner



> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>