[erlang-bugs] EUnit treats a process that kills itself as a successful test

[Richard Carlsson]

John Hughes wrote:
> Here's my code:
>  
> -module(eunit_example).
> -include_lib("eunit/include/eunit.hrl").
>  
> exit_test() ->
>     exit(self(),die).
>  
> In the shell, I run:
>  
> 1> c(eunit_example).
> {ok,eunit_example}
> 2> eunit_example:test().
>   Test successful.
> ok
> 
> Is that really the intention?
>  
> Likewise, this test passes:
>  
> spawn_test() ->
>     spawn_link(erlang,exit,[dying]),
>     timer:sleep(1).
>  
> (The sleep is there to allow time for the child process to die, and the
> exit signal to be propagated).
>  
> Presumably the process running the test is trapping exits--but is that
> really appropriate? As in this last example, crashes in child processes
> won't cause the test to fail.

At some point, I had the idea that it would be good to make the test
processes default to trapping exits, to make it easier to write some
kinds of process-spawning tests. (There was a comment to this effect
in the code, but it was never documented.) This was probably misguided,
so I have now checked in a change that makes the test processes
non-trapping, which will probably cause less astonishment.
(Trapping can still be enabled by the user where necessary).

One thing to remember, though, is that unless you wrap every single
test in a separate {spawn,Test}, a test process will run several
tests, one at a time. (In the simplest case only one test process
is ever spawned, and runs all the tests.) Hence, if the process dies,
this will not simply "fail" a single test, but cause all the tests that
were to be executed by that process to instead be cancelled/skipped.
Hence, if a test (or group of tests) is known to be at risk of
receiving an exit signal, it is best to wrap it in {spawn, Test},
so that the effect is isolated.

    /Richard

-- 
 "Having users is like optimization: the wise course is to delay it."
   -- Paul Graham