[erlang-questions] Testing data with subsets using QuickCheck

[Krzysztof Jurewicz]

Let’s say that we want to test lists:member/2 using QuickCheck/triq/PropEr. We want to ensure that:

For every element and a list such that the element is member of the list, lists:member/2 returns true.

Written as a QuickCheck property, this may look like:

prop_member() ->
    ?FORALL(
       {List, Member},
       list_with_member(),
       lists:member(Member, List)).

The problem is that there is no list_with_member/0 generator. A few solutions are possible:

⒈ Test all list members instead of one.

prop_member() ->
    ?FORALL(
       List,
       list(int()),
       lists:all(
         fun (Member) -> lists:member(Member, List) end,
         List)).

This may look like checking a bit too much in one test iteration.

We can also write list_with_member/0 generator in one of the following ways:

⒉ Choose one member randomly.

list_with_member() ->
    ?LET(
       {List, Seed},
       {non_empty(list(int())), {int(), int(), int()}},
       begin
           _ = rand:seed(exs64, Seed),
           Index = rand:uniform(length(List)),
           {List, lists:nth(Index, List)}
       end).

This doesn’t look very elegantly and raises questions about whether it will shrink gracefully.

⒊ Insert an element into generated list.

list_with_member() ->
    ?LET(
       {List, Element},
       {list(int()), int()},
       {[Element|List], Element}).

The problem with this approach in this particular case is that an element is always inserted at the beginning of the list, so we won’t get any errors for implementations like this one:

member(Element, [Element|_]) ->
    true;
member(_, _) ->
    false.

⒋ Write a custom generator.

This seems to be generally discouraged.

Any thoughts about which of the approaches above (or maybe yet another one) should be recommended in similar cases?

Krzysztof