[erlang-questions] Killer Erlang tools

[Ulf Wiger (TN/EAB)]

Joel wrote:
> By specification I meant something declarative (what) as 
> opposed to instructions on how  to test stuff (how) :-). I 
> think only QC allows that.

That's a matter of interpretation, I think.

While I agree that QC is the only tool that 
lets you lay down the rules for testing rather
than describing each individual test, even
plain test cases in Erlang are declarative
by nature.

For example, when using the OTP test server,
you write down test cases as individual functions
(one test case - one function), and the test
succeeds if the function returns ok; everything
else is considered a failure. The test case itself
usually relies heavily on (declarative) pattern
matching. An example:

rdbms_unique_ix(doc) ->
    "Test the 'unique' option for indexes";
rdbms_unique_ix(suite) ->
  [];   % there are no sub-tests
rdbms_unique_ix(Config) when is_list(Config) ->
    ?line {atomic,ok} = rdbms:create_table(
                          u_ix,
                          [{ram_copies, [node()]},
                           {attributes, [key, value]},
                           {rdbms, 
                            [
                             {indexes, 
                              [{{value,value},?MODULE,ix_value,[],
                                [{type,set},
                                 {unique, true}]}]}
                            ]}
                          ]),
    ?line trans(fun() ->
                        mnesia:write({u_ix,1,a}),
                        mnesia:write({u_ix,2,b})
                end),
    ?line trans(fun() ->
                        [{u_ix,1,a}] = 
                            mnesia:index_read(u_ix,a,{value,value}),
                        [{u_ix,2,b}] = 
                            mnesia:index_read(u_ix,b,{value,value})
                end),
    ?line fail_trans(fun() ->
                             mnesia:write({u_ix,3,a})
                     end).
 
The ?line macros are optional, and are simply 
utility macros to help identify exactly where the 
test went wrong. The helper functions trans/1 and 
fail_trans/1 will exit if passing the argument as 
a fun to mnesia:transaction/1 commits or aborts 
respectively, thus acting as assertions.

This is actually quite similar to how one would go
about writing QC tests, but with the main difference
that you work with symbolic input, and then let QC
generate loads of different input values.

Here's an example using eUnit:

-module(fib).
-export([fib/1]).
-include_lib("eunit/include/eunit.hrl").

fib(0) -> 1;
fib(1) -> 1;
fib(N) when N > 1 -> fib(N-1) + fib(N-2).

fib_test_() ->
    [?_assert(fib(0) == 1),
     ?_assert(fib(1) == 1),
     ?_assert(fib(2) == 2),
     ?_assert(fib(3) == 3),
     ?_assert(fib(4) == 5),
     ?_assert(fib(5) == 8),
     ?_assertException(error, function_clause, fib(-1)),
     ?_assert(fib(31) == 2178309)
    ].

It's also pretty declarative in that you
focus on what is supposed to happen, but it's 
of course "pin-prick testing", compared to 
QC, which is much closer to model-checking,
but much easier to get into.

If you really want to get into esoteric stuff
(and I'm sure you do), you can look at some of 
the stuff done by Thomas Arts et al. In

http://www.ituniv.se/~arts/papers/fates04.pdf

they describe how they went about testing a 
leader election implementation, which had first 
been verified using model checking, then further
tested using abstract traces (+ model checking),
which revealed serious bugs; then the code was 
reimplemented and verified with abstract traces,
but QuickCheck still found faults in the tested
code.

If nothing else, it illustrates what a thankless
job software testing is. (:

(If you want to explore model checking for Erlang,
then McErlang ought to be someting to consider:
http://babel.ls.fi.upm.es/~fred/McErlang/)

BR,
Ulf W

> -----Original Message-----
> From: Joel Reymont [mailto:joelr1@REDACTED] 
> Sent: den 13 juni 2007 13:48
> To: Ulf Wiger (TN/EAB)
> Cc: Tim Uckun; Erlang Questions
> Subject: Re: [erlang-questions] Killer Erlang tools
> 
> By specification I meant something declarative (what) as 
> opposed to instructions on how  to test stuff (how) :-). I 
> think only QC allows that.
> 
> On Jun 13, 2007, at 12:33 PM, Ulf Wiger (TN/EAB) wrote:
> 
> > 3) Erlang QuickCheck (http://www.quviq.com)
> >
> 
> --
> http://topdog.cc      - EasyLanguage to C# translator
> http://wagerlabs.com  - Blog
> 
> 
> 
> 
> 
>