[erlang-questions] Testing a large, heterogeneous system

[Motiejus Jakštys]

On Wed, Jul 31, 2013 at 11:55 AM, David Welton <davidnwelton@REDACTED> wrote:
>> Without knowing the internals of your system it's hard to give specific
>> advice, but if you have a bunch of state-full stuff proper [2]
>> has some very powerful tools for testing states and reasoning about what
>> should be valid before and after state transitions. Checkout proper_statem
>> [3] and this [4] example of a very useful test implementing the
>> proper_statem callbacks.
>
> Looks interesting - are there any examples?  It looks a bit ...
> abstract.  With limited time and Erlang abilities, we need to aim for
> stuff that gets the most coverage with the least amount of work.

PropEr (and quickcheck, and all the property based testing) is
certainly not easy, but really, really pays off. Check out the
paper[1] (it's an easy read) and tutorials[2]. I've been working on
putting all I can find projects that use PropEr to a single place for
kind-of CI tests. Still in progress..

For instance, we tested our distributed CRUD system. PropEr interacts
with it (mess with the database, add/remove nodes), and checks whether
our postconditions hold (for instance, you get back what you have
written). We have found numerous cache coherency issues, race
conditions. The fun really started when we started turning on/off the
worker nodes during the test.

Hope that helps.
Motiejus

[1]: http://proper.softlab.ntua.gr/papers/proper_types.pdf
[2]: http://proper.softlab.ntua.gr/Tutorials


-- 
Motiejus Jakštys