[erlang-questions] I Hate Unit Testing...
Nick Gerakines
nick@REDACTED
Wed Jan 28 00:54:48 CET 2009
I think you'll find that you aren't alone. I've talked to several
people who feel the same way and ultimately came up with my own
solution.
A short while back I took over an Erlang project called etap with the
purpose of creating a unit testing facility that produces TAP output.
It gives you more flexibility when writing unit tests because your
test code is executed in whichever order you write it in as long as
your tests match your plan.
The project is open source under the MIT license and is available on
GitHub. We are using it regularly here at EA and have been actively
updating the project in regular intervals.
http://github.com/ngerakines/etap
You can also use it to create code coverage reports and do things like
pipe your test results into Perl's TAP::Harness module. Several of my
open source projects (like erlang_protobuffs) use it and working
examples can be found there.
Hope that helps a bit.
# Nick Gerakines
On Tue, Jan 27, 2009 at 3:37 PM, Steve Davis
<steven.charles.davis@REDACTED> wrote:
> Hi all,
>
> I really hate writing code to test my code. I have noticed that most
> people must also hate writing test code since the unit test code
> included in almost every project I've ever seen is... well, scanty, at
> best.
>
> But it came to me that *there's a way out of this* thanks to Erlang, since:
> - all functions in Erlang are deterministic, i.e. for a certain input
> you are guaranteed one and only one output, and so a simple MFA apply
> should always give a predictable result.
> - reading in terms to feed to apply/3 is trivial.
>
> So I got to hacking something together today to see how/if this idea
> would work. I'm posting the results of this experiment here to see if
> anyone thinks the idea is interesting and has legs. Hopefully I may get
> a few pointers on improving my naive and diabolical code as I'm
> definitely a "learner".
>
> Familiarity is utmost so I just borrowed/subverted the term structure
> used for .app files, like this...
> ----- my_app.test ------
> %% utest test specification
> {utest, my_app,
> [{description, ""},
> {version, "0.1"},
> {flags, [verbose]}, %% only [verbose] or [] supported
> {applications, [kernel, stdlib, crypto]}, %% dependencies
> {tests, [
> % Tests are specified as Module, Function, ArgumentList, ExpectedResult
> % i.e. {M, F, A, R}
> {my_module, hex, [2], "2"},
> {my_module, hex, [255], "ff"},
> {my_module, hex, [55551], "d8ff"}
> ]}
> ]}.
>
> ...after a few hours hacking around I had a simple demo module that
> produces output like this...
> ---- Sample Console Session ----
> 3> utest:run("../test/my_module.test").
> UNIT TEST
> Application: my_app
> Version: "0.1"
> Description: ""
> Unit Tests: 3
> Depends on: [kernel,stdlib,crypto]
> Starting crypto...ok
> Running tests...
> [FAIL] my_module:hex(2) -> 50 EXPECTED "2"
> [OK] my_module:hex(255) -> "ff"
> [OK] my_module:hex(55551) -> "d8ff"
> ***FAILURE: There were 1 test failures
> {fail,{tests,3},{errors,1}}
> 4>
>
> ...now I'll grit my teeth and post the code in case anyone wishes to try
> it out (or help me by giving me style pointers - which I know I need!)
> ---- utest.erl ----
> %% Simple Unit Testing without writing code!
> -module(utest).
> -vsn("0.1").
> -author('steven.charles.davis@REDACTED').
>
> -record(utest, {package="unknown", version="0.0", description="",
> flags=[], depends=[], tests=[]}).
>
> -export([run/1]).
>
> %% File is a ".test" spec file
> run(File) ->
> {ok, [Utest]} = file:consult(File),
> {utest, Package, Options} = Utest,
> R = #utest{package=Package},
> R1 = get_options(Options, R),
> io:format("UNIT TEST~n Application: ~p~n", [R#utest.package]),
> N = length(R1#utest.tests),
> io:format(" Version: ~p~n Description: ~p~n Depends on: ~p~n Unit
> Tests: ~p~n",
> [R1#utest.version, R1#utest.description, R1#utest.depends, N]),
> ensure(R1#utest.depends),
> case R1#utest.flags of
> [verbose] -> V = true;
> _ -> V = false
> end,
> io:format("Running tests...~n", []),
> {X, E} = do_tests(R1#utest.tests, 0, V),
> {X, {tests, N}, {errors, E}}.
>
> %%
> get_options([H|T], R) ->
> case H of
> {version, Value} -> R1 = R#utest{version=Value};
> {description, Value} -> R1 = R#utest{description=Value};
> {flags, Value} -> R1 = R#utest{flags=Value};
> {applications, Value} -> R1 = R#utest{depends=Value};
> {tests, Value} -> R1 = R#utest{tests=Value};
>
> _ -> R1 = R
> end,
> get_options(T, R1);
> get_options([], R) ->
> R.
>
> %%
> ensure(Apps) ->
> ensure(Apps, application:which_applications()).
> %%
> ensure([App|Rest], Running) ->
> case lists:keysearch(App, 1, Running) of
> {value, {App, _, _}} ->
> ok;
> false ->
> io:format("Starting ~p...", [App]),
> ok = application:start(App),
> io:format("ok~n", [])
> end,
> ensure(Rest, Running);
> ensure([], _) ->
> application:which_applications().
>
> %%
> do_tests([H|T], E, V) ->
> {M, F, A, R} = H,
> R1 = apply(M, F, A),
> S = lists:flatten(io_lib:format("~w", [A])),
> S1 = string:sub_string(S, 2, length(S) - 1),
> case R1 of
> R ->
> E1 = E,
> case V of
> true -> io:format(" [OK] ~p:~p(" ++ S1 ++ ") -> ~p~n", [M, F, R1]);
> false -> ok
> end;
> _ ->
> E1 = E + 1,
> io:format(" [FAIL] ~p:~p(" ++ S1 ++ ") -> ~p EXPECTED ~p~n", [M, F,
> R1, R])
> end,
> do_tests(T, E1, V);
> do_tests([], E, V) ->
> case V of
> true when E =:= 0 -> io:format("SUCCESS! All tests completed
> successfully~n", []), {ok, E};
> true when E > 0 -> io:format("***FAILURE: There were ~p test
> failures~n", [E]), {fail, E};
> false when E =:= 0 -> {ok, E};
> false when E > 0 -> {fail, E};
> _ -> {error, E}
> end.
>
> Well, that's the basic idea -- there's a heap of improvements that could
> be made of course, but thanks for reading this far, I look forward to
> any comments you may have!
>
> BR,
> /scd
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
More information about the erlang-questions
mailing list