[erlang-questions] PropEr and determinism of statem scenario execution

[Scott Lystig Fritchie]

Paul Peregud <paulperegud@REDACTED> wrote:

pp> Hi everyone!  This is a PropEr-related question. [...]

pp> The problem - I don't have a control over schedulers. That leads to
pp> scenarios that MAY OR MAY NOT fail. And that causes problems with
pp> shrinking. As a result after a night of crunching scenarios on a
pp> fast machine I get a test case of 70 steps that is impossible to
pp> debug, because it is too large. When I repeat this test case using
pp> proper:check/2, it fails very seldom.

Just guessing here, but it sounds like you're describing a situation
where there's a side-effect that your model isn't accounting for.  That
thing is usually time, either using erlang:now() or os:timestamp() or
erlang:time() or something like that.

State that happens to be lying around in the file system that affects
your code is also a fun source of nondeterminism.

You can try something like this (untested):

L = [begin
        {_, Before, _} = now(),
        Res = (catch true = proper:check(Property, CounterExample)),
        {_, After, _} = now(),
        {After - Before, Res}
     end || _ <- lists:seq(1,400)].
FailL = [Secs || {Secs, Res} <- L, Res /= true].

... and then see how many items in FailL cross a wall-clock-second-tick
boundary.  It helps if the normal execution time is less than 1 second.
:-)

The only real solution is find all sources of nondeterminism, wall-clock
or whatever source, and stamp them out.

-Scott