[erlang-questions] Trace-Driven Development

[Michael Turner]

I'm a little embarrassed to be doing a "reply all" on this message,
because I'm (still) such a stop-start Erlang newbie. What compensates
for the mortification, however, is passages like the following, which
suggest I'm hardly the only one who should be embarrassed:

"Erlang tracing is a seething pile of pain that involves reasonably
complex knowledge of clever ports, tracing return formats, and
specialized tracing MatchSpecs (which are really their own special
kind of hell). The tracing mechanism is very powerful indeed, but it
can be hard to grasp."

Obviously, that kind of statement has no place in the official
documentation of a professional product. Oh, except that's precisely
where I found it:

  http://www.erlang.org/doc/apps/et/et_intro.html#id62156

Now, if the formal documentation hosted by Ericsson seems determined
to frighten me away, I'll oblige Ericsson and run off. That "hell"
paragraph immediately followed two paragraphs that I couldn't make
much sense of, so it pretty much validated my confusion and dismay at
that point, while warning me that it was only going to get worse.

[I wrote 5 more lines of rant here, then deleted them in the interests
of being diplomatic.]

I started using seq_trace. It has its own documentation problems, of
course. For example: since seq_trace is an implementation of Lamport
clocks, you should *say* somewhere (like, in the first paragraph,
maybe?) that it's an implementation of Lamport clocks. Don't make it
sound like your own invention. That's dishonest. And don't make people
infer it. That's wasting people's time. Just say it. It only takes a
few seconds to type "seq_trace implements Lamport clocks."

Still, I found seq_trace relatively simple and usable, and I'm now
doing unit testing on a module as I develop it further, based on
collecting and filtering seq_trace results with a small amount of code
I wrote myself. At some point, I expect to finally put the horse in
front of the cart and do *trace*-driven development. But before then,
I should make a decision: do I keep building ever more sophisticated
match filtering on top of seq_trace, undoubtedly reinventing wheel
after wheel, or do I bite the bullet and plunge into what Jayson
Vantuyl describes as "hell"?

It's a discouraging choice.

-michael turner

On Sun, Jun 3, 2012 at 4:02 AM, Henning Diedrich <hd2010@REDACTED> wrote:
> Should there be any interest I would like to discuss tracing and if it may
> be more valuable for the actor model than unit tests.
>
> A couple of days ago, during Jesper Louis Andersen's very enlightening
> debugging and tracing tutorial at Ericsson, Joe Armstrong asked whether
> showing visual traces first may be a good approach to teaching Erlang,
> book-side.
>
> I would like to propose again that trace-driven development may possibly be
> the way to teach andalso program Erlang.
>
> In the instance, we were talking about visual traces, like these:
>
> http://www.erlang.org/doc/apps/et/coffee_order.png
>
> I mean it in the slightly unrealistic way that most people probably
> appreciate test-driven programming as a great idea but seldom execute it as
> dogmatically as preached.
>
> But I realize that what I actually do when programming a new app, or when
> looking for truly nasty bugs, is almost always pretty 'trace-driven'.
> Writing traces into the program has become a routine step for me when coding
> medium complicated stuff that uses supervisors, monitors and multiple
> process that come to live at various times throughout the lifetime of an
> Erlang (OTP) application -- and most of all, during the start up. And I have
> always found surprises, which process comes to live when. I think Jesper
> related a similar story how they found tons of bugs that no-one had realized
> were there when using tracing (not sure any more.)
>
> I believe that the main aggravating factor is that you don't construct an
> OTP application from scratch that often. You'll have to look things up again
> every next time. My guess is that this is true for the vast majority of
> Erlang programmers. Of course, for the ultimate expert, it may be hard to
> grasp what I am even talking about because it's all so 'obvious'.
>
> But what I really find myself currently doing, before writing tests, is
> writing traces. I also haven't found out how to plan out and write
> meaningful stateful tests up front, before I even made it to architect the
> application.
>
> My concrete order is: writing first, get to compile, put in traces, try to
> run, debug.
>
> I am so sure meanwhile that I will need the traces that I don't wait anymore
> until I have gotten lost in searching for a bug and not even knowing what
> happens before the crash and what parts of the program may have silently
> died and what other parts may have waited for which other part.
>
> These things will be obvious for expert Erlang programmers who have set up
> applications time and again. But few people using Erlang for productivity
> reasons will ever come into that position. And for finding bugs: that's
> exactly the moment when reality doesn't follow your fantasy and a reality
> check in the form of a trace is one of the first tools to turn to. Is it
> not?
>
> So trace-driven development sounds like it could be a useful recommendation.
> And if it is, a new best practice of waterfalling your application into
> existence could maybe emerge, similar to what Joe describes in his book (if
> I remember well) how he almost always starts out.
>
> I learned a lot in Jesper's tutorial. For example that because Erlang and
> OTP keep growing, there are now three distinct mechanisms, with respective
> modules and function calls, to trace and debug. And despite me using traces
> a lot, I hadn't used any of them, for the wrong reasons it turned out. (And
> for productivity, a simple io:format can beat getting tied up into using
> more powerful but less simple minded options.)
>
> So I am asking this out of honest ignorance:
>
> Instead of tracing, is there a way to write meaningful, concise tests that
> are meant or at least good for checking the sequence of process
> communications? In other words, are there better alternatives to follow
> traces with the naked eye? I do not mean a heavy construct, or an
> also-possible use, but a test-package made for that and/or a useful practice
> that someone is in fact applying?
>
> And it need not be the visual traces as the image linked to above. A
> ping-pong output in the terminal may do (if not skewered by delays or
> re-sequences or contentions in io:format.)
>
> For trace-driven development, what concrete procedure with which concrete
> calls and parameters to erl can be a best practice for starting out? I am
> sure it' s 'obvious' for many, but at least for me, not. What could be the
> hello world case for a trace-driven approach?
>
> And finally, how do you approach setting up a new app, finding a bug -- how
> much tracing does everyone use in these things? And is anyone also using
> io:format instead of the OTP goodness, if so, for a good reason?
>
> Best,
> Henning
>
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>