[erlang-questions] Trace-Driven Development

[Joe Armstrong]

On Sat, Jun 2, 2012 at 9:02 PM, Henning Diedrich <hd2010@REDACTED> wrote:
> Should there be any interest I would like to discuss tracing and if it may
> be more valuable for the actor model than unit tests.
>
> A couple of days ago, during Jesper Louis Andersen's very enlightening
> debugging and tracing tutorial at Ericsson, Joe Armstrong asked whether
> showing visual traces first may be a good approach to teaching Erlang,
> book-side.
>
> I would like to propose again that trace-driven development may possibly be
> the way to teach andalso program Erlang.
>
> In the instance, we were talking about visual traces, like these:
>
> http://www.erlang.org/doc/apps/et/coffee_order.png

These are great.

If I have a complex protocol to understand I often draw these (on
paper) but then
throw them away after the code has been written (the throwing them
away bit is wrong I know)
The problem is "published code" that ends up in books git repositories
does not contain all
the design notes used to create the code - not is there a clue as to
how the code was created.

(I'm thinking about ways to bundle "research" and code toghether so
that *both* get published)

Message sequence charts (MSCs) are parts of SDL and UML and there are many tools
(which I don't use)  to make them - a quick google turned up this
http://en.wikipedia.org/wiki/MscGen which is food for thought.

Using the trace BIFs it should be easy (famous last words) to make a
checker that
checks if the observed messages correspond to allowed flows in the MSC

This strikes me as a abstraction-lift over unit tests ...

>
> I mean it in the slightly unrealistic way that most people probably
> appreciate test-driven programming as a great idea but seldom execute it as
> dogmatically as preached.
>
> But I realize that what I actually do when programming a new app, or when
> looking for truly nasty bugs, is almost always pretty 'trace-driven'.
> Writing traces into the program has become a routine step for me when coding
> medium complicated stuff that uses supervisors, monitors and multiple
> process that come to live at various times throughout the lifetime of an
> Erlang (OTP) application -- and most of all, during the start up. And I have
> always found surprises, which process comes to live when. I think Jesper
> related a similar story how they found tons of bugs that no-one had realized
> were there when using tracing (not sure any more.)
>
> I believe that the main aggravating factor is that you don't construct an
> OTP application from scratch that often. You'll have to look things up again
> every next time. My guess is that this is true for the vast majority of
> Erlang programmers. Of course, for the ultimate expert, it may be hard to
> grasp what I am even talking about because it's all so 'obvious'.
>

I for one have to look things up a lot of the time - the difference is
that I know
exactly where to look.


> But what I really find myself currently doing, before writing tests, is
> writing traces. I also haven't found out how to plan out and write
> meaningful stateful tests up front, before I even made it to architect the
> application.

That's what I'd do - I'd write the traces first.

Possibly http://www.mcternan.me.uk/mscgen/ could be integrated with
this work flow

I'm not sure about testing the state. I just want to see what is in the messages

>
> My concrete order is: writing first, get to compile, put in traces, try to
> run, debug.
>
> I am so sure meanwhile that I will need the traces that I don't wait anymore
> until I have gotten lost in searching for a bug and not even knowing what
> happens before the crash and what parts of the program may have silently
> died and what other parts may have waited for which other part.
>
> These things will be obvious for expert Erlang programmers who have set up
> applications time and again. But few people using Erlang for productivity
> reasons will ever come into that position. And for finding bugs: that's
> exactly the moment when reality doesn't follow your fantasy and a reality
> check in the form of a trace is one of the first tools to turn to. Is it
> not?
>
> So trace-driven development sounds like it could be a useful recommendation.
> And if it is, a new best practice of waterfalling your application into
> existence could maybe emerge, similar to what Joe describes in his book (if
> I remember well) how he almost always starts out.
>
> I learned a lot in Jesper's tutorial. For example that because Erlang and
> OTP keep growing, there are now three distinct mechanisms, with respective
> modules and function calls, to trace and debug. And despite me using traces
> a lot, I hadn't used any of them, for the wrong reasons it turned out. (And
> for productivity, a simple io:format can beat getting tied up into using
> more powerful but less simple minded options.)
>
> So I am asking this out of honest ignorance:
>
> Instead of tracing, is there a way to write meaningful, concise tests that
> are meant or at least good for checking the sequence of process
> communications? In other words, are there better alternatives to follow
> traces with the naked eye? I do not mean a heavy construct, or an
> also-possible use, but a test-package made for that and/or a useful practice
> that someone is in fact applying?
>
> And it need not be the visual traces as the image linked to above. A
> ping-pong output in the terminal may do (if not skewered by delays or
> re-sequences or contentions in io:format.)
>
> For trace-driven development, what concrete procedure with which concrete
> calls and parameters to erl can be a best practice for starting out? I am
> sure it' s 'obvious' for many, but at least for me, not. What could be the
> hello world case for a trace-driven approach?

Good question. You'd have to make a mini-language to describe message sequences
something like TCL expect

    [{send,client,server,hello},                %% client sends a
hello message to server
     {receive,client,server,ack}               %% cleint receives an
ack from server
   ]

For such a spec one could generate a PNG and test code :-)
(There is a library for making PNGs well hidden in the erlang distribution
code:which(egd) will find it for you)

(aside: there are lots of good things in the erlang distribution which
nobody knows about
like egd.erl -- one of lifes mysteries is why these are not documented
or promoted -
I suspect lack of time and "not core businesss")

>
> And finally, how do you approach setting up a new app, finding a bug -- how
> much tracing does everyone use in these things? And is anyone also using
> io:format instead of the OTP goodness, if so, for a good reason?

Absolutely - I use io:format all the time. I put the io:formats next
to send and receive statements.

The good reason was "I didn't know about the trace bifs"
(modified truth) - I knew about them but hadn't realized how good they
were until
I attended Jespers (excellent) tutorial. I'd relegated traces to an
obscure appendix in my
book (Appendix E.3) where nobody could find it.

In the next edition (Which I'm working on *now*) I will introduce
tracing far far earlier


Cheers

/Joe

>
> Best,
> Henning
>
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>