Sequence does't apply to sub-groups in common tests?

[Dawid Figiel]

Same here.
If anyone wants to try out what we're talking about, try the following:

Code:
-module(test_SUITE).

-export([all/0,
    groups/0,
    init_per_suite/1,
    end_per_suite/1,
    init_per_group/2,
    end_per_group/2,
    init_per_testcase/2,
    end_per_testcase/2]).

-export([failing_tc/1, ok_tc/1]).

all() ->
    [{group, test}].

groups() ->
    [{failing_group, [], [failing_tc]},
     {ok_group, [], [ok_tc]},
     {test, [sequence], [{group, failing_group}, {group, ok_group}]}].

init_per_suite(Config) -> Config.
end_per_suite(_Config) -> ok.

init_per_group(_Group, Config) -> Config.
end_per_group(failing_group, _Config) -> {return_group_result, failed};
end_per_group(_Group, _Config) -> ok.

init_per_testcase(_TestCase, Config) -> Config.
end_per_testcase(_TestCase, _Config) -> ok.

failing_tc(_Config) -> 2=3.
ok_tc(_Config) -> ok.


running using:
>run_test -suite test_SUITE -logdir <logdir>
(run_test as described in
http://www.erlang.org/doc/apps/common_test/run_test_chapter.html#id2267273)

it should give (in short):
Code:
- - - - - - - - - - - - - - - - - - - - - - - - - -
test_SUITE:failing_tc could not be executed
Reason: {badmatch,3}
- - - - - - - - - - - - - - - - - - - - - - - - - -

Testing Erlang.group_test.test_SUITE: *** FAILED *** test case 1 of 2
Testing Erlang.group_test.test_SUITE: TEST COMPLETE, 1 ok, 1 failed of 2
test cases


*Even though the second tc should be skipped.*




On Mon, Jul 12, 2010 at 3:17 PM, Dawid Figiel <dawid.figiel@REDACTED>wrote:

> Hi,
>
> I need some feedback, why sequence doesn't apply to sub-groups in common
> tests?
>
> example:
>
> This works:
>
> groups() -> [{group1, [sequence], [test1,
>                                    test2]}].
>
> If test1 fail then test2 will be skipped.
>
> And this does not work:
>
> groups() -> [{group1, [sequence], [{subgroup1, [], [test1]},
>                                    {subgroup2, [], [test2]}]}]
>
> end_per_group(Group, Config) ->
>     Status = ?config(tc_group_result, Config),
>     case proplists:get_value(failed, Status) of
>         [] -> {return_group_result, ok};
>         _Failed -> {return_group_result, failed}
>     end.
>
> Test1 has failed, so subgroup1 is handled as a failed too, but subgroup2 is
> not skipped although we have set up 'sequence'...why ?
>
> According to documentation:
>
> "A failed sub-group (Status == failed) will cause the execution of a
> sequence to fail in the same way a test case does."
>
>
> Any clues ?
>
> --
> regards,
>
> Dawid
>



-- 
Dawid