[erlang-questions] EUnit not failing test cases which timeout

[André Graf]

Hi Ignas

You can use the timeout control in eunit and specify how long your
testcases may take to complete e.g.

-module(foobar).
-include_lib("eunit/include/eunit.hrl").
timing_out_test_() ->
    {
        setup, fun() ->ok end,
        [
            {timeout, 1, fun() -> timer:sleep(10000) end}
        ]
    }.

In the example above your testcase may not take longer than one
second, since it obviously does it will fail. I don't know another way
than using a test suite tough, maybe there are other hints on that.

Cheers,
Andre


2012/3/6 Ignas Vyšniauskas <baliulia@REDACTED>:
> Hi,
>
> if I have a test case which is timing out for some reason, EUnit will
> not consider that as a failure, i.e. a module like this:
>
> -module(foobar).
> -include_lib("eunit/include/eunit.hrl").
>
> timing_out_test() ->
>    timer:sleep(10001).
>
> will produce this:
>
> ======================== EUnit ========================
> foobar: timing_out_test (module 'foobar')...*timed out*
> undefined
> =======================================================
>  Failed: 0.  Skipped: 0.  Passed: 0.
> One or more tests were cancelled.
> ERROR: One or more eunit tests failed.
>
> As you can see in the last 3 lines, there is a bunch of information
> which contradicts itself. Terrible!
>
> On the other hand, eunit_surefire will generate a report which claims
> the test was skipped:
>
> <testsuite tests="1" failures="0" errors="0" skipped="1" time="0.000"
> name="module 'foobar'">
>  <testcase time="0.000" name="foobar:timing_out_test/0_0"
> description="module 'foobar'">
>    <skipped type="timeout"/>
>    <system-out>
>    </system-out>
>  </testcase>
> </testsuite>
>
> Why is timing out being considered a skipped test case rather than a
> failed one? Time-out's happen quite a lot in Erlang, but due to this,
> one might not experience the failure. I believe this is wrong and tests
> which time-out should be marked as failures.
>
> It would be good to have any feedback on this.
>
> Thanks,
> Ignas
>
> P.S. Maybe I should post this to erlang-bugs instead?
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions