[erlang-questions] Quickcheck/PropEr: List generator not working as expected

[Kostis Sagonas]

On 10/19/11 15:29, Ward Bekker wrote:
> See git@REDACTED:wardbekker/proper_test.git , src/test.erl for a runnable demo app. Start with ./shell from the projects root
>
> When I test the property `proper:quickcheck(test:prop_ends_with_double_cons_is_true()).` The following error is outputted to console:
>
> Error: Couldn't produce an instance that satisfies all strict constraints after 50 tries.
>
> Is seems that `?SUCHTHAT(Drow, list(consonant()), length(Drow)>  2)` can't generate valid values.
>
> When I output the values generated by `list(consonant())` I notice that only zero or one length lists are generated. Why is that?

First of all, regarding PropEr, you may be getting better/faster answers 
if you use the e-mail address in the cc: of this mail instead of the 
erlang-questions. This is because only few of the PropEr developers read 
this list.

Now, concerning your question, you already got some answers that helped 
you fix the problem you are facing, but I am not sure they are the 
proper (TM) answers, so let me also give it a try.

I see that in your program you have the following:

-define(VOWELS, [$a, $e, $i, $o, $u, $y]).

So far so good. Then you define the following:

-spec consonant() -> proper:test().
consonant() ->
     ?SUCHTHAT(Char, char(), lists:all(fun(C) -> Char /= C end, ?VOWELS)).

Let mention that the above:
  1. does not define a proper test; instead, it defines a generator
  2. seems a complicated (and a bit weird) way of writing a definition 
for consonants...
However, these remarks are besides the point here.

What you should keep in mind about generators is that they are *lazy*. 
This is very important here. What this means is that they do not get 
evaluated to some concrete value (say $c) but instead they produce a 
closure which knows how to generate (new) values when needed. So, your 
use in:

    ?SUCHTHAT(Drow, list(consonant()), length(Drow) >  2)

basically does the following:

  1. It sees the consonant() and list(consonant()) generators and 
generates symbolic terms for them.

  2. To be able to see whether the term generated for list(consonant()) 
satisfies the length(Drow) > 2 condition, the generator gets evaluated. 
Let's assume that the term that the consonant() generator gets evaluated 
is $c. Then a list($c) will be generated and since lists start small, 
these are quite often the empty list [] or the list [$c] with just one 
element.

  3. Since for the above the ?SUCHTHAT condition is not satisfied, 
PropEr will try to generate a new term for the list(consonant()) generator.

  4. Now, you may think that hey, the consonant() generator has already 
been evaluated to a value, namely $c, so a new list of $c's will be 
generated next, and since the empty list and the one element list have 
already been generated, a bigger list of $c's will be generated next. 
However, that's NOT what's happening here!

  5. Instead, the consonant() generator is re-evaluated so a new term 
gets produced, let's say $d, and a new list($d) generator will be 
evaluated next.

Now, I hope that you can see where this is heading...  list($d) is a 
totally new generator and has similarly equal probability of generating 
a small list (empty or just one element) as the previous generator 
(list($c)) had.

Anyway, the point is that the consonant() generator may look like an 
Erlang function, but it's not evaluated to a value once and for all. 
Instead, it's evaluated lazily.  You should keep this in mind because 
it's a subtle point in an otherwise eager language like Erlang.

Hope this helps,
Kostis