[erlang-questions] Re: False negatives in EUnit

[Simon Thompson]

Per - yes, I have been looking at EUnit tests / refactoring - together with Thomas Arts and Huiqing Li -  and one option that we discussed was to indicate potential errors of this sort: things like this are not strictly "illegal" but probably worth checking in more detail. Of course, it's difficult to spot all cases, but the mismatches here make plain that something untoward is probably happening.

I guess they are in fact false *positives* - tests that pass which shouldn't: the worst kind of erroneous test.

Regards

Simon


On 3 Apr 2011, at 17:40, Per Melin wrote:

> Below are five EUnit tests that all pass without so much as a warning.
> If you're an experienced EUnit user you will tell me that they are all
> illegal tests, which they are. But is it obvious? If you are less
> experienced with EUnit, or maybe just a little clumsy, or go too fast
> when you refactor a test you could write something like this, and you
> may never know. All it takes is that you fail to spot the missing or
> extra underscore.
> 
> To silently hide errors is the last thing I want in a testing
> framework. Is there a way to fix this?
> 
> a_test() ->
>    ?_assertEqual(1, 2).
> 
> b_test() ->
>    ?_test(?assertEqual(1, 2)).
> 
> c_test_() ->
>    ?_test(fun() -> 1 = 2 end).
> 
> d_test_() ->
>    [fun() -> ?_assertEqual(1, 2) end].
> 
> e_test() ->
>    {setup,
>        fun() -> ok end,
>        [?_assertEqual(1, 2)]}.
> 
> 
> ======================== EUnit ========================
> module 'oops'
>  oops:12: c_test_...ok
>  oops: a_test...ok
>  oops: b_test...ok
>  oops: d_test_...ok
>  oops: e_test...ok
>  [done in 0.017 s]
> =======================================================
>  All 5 tests passed.
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions