[erlang-questions] Automated testing of a distributed application

[Garret Smith]

I just finished setting up an automated test suite for my distributed
application using the common_test "Large Scale Testing" support.  It was
rough, but *almost* works.  It starts up 2 local nodes, starts my app on
both nodes, runs a few test cases from my suite on node1, a second batch of
cases on node2, then yet another batch of cases on node1.

The test cases are designed to work by contacting the distributed app on
node1, modifying it's state through the API, then contacting the app on
node2 to verify that the state changes are reflected there, then repeating
from the perspective of node2.

sample test spec, invoked with ct_master:run/1

{logdir, "../logs"}.

{alias, testdir, "."}.

{init, [n1, n2], {node_start, [{erl_flags, "-pa ../../ebin"},
{monitor_master, true}]}}.

{merge_tests, false}.

{cases, n1, testdir, distributed_SUITE, [create_table, insert_1]}.
{cases, n2, testdir, distributed_SUITE, [read_1, update_2]}.
{cases, n1, testdir, distributed_SUITE, [read_2]}.



Now that my basic setup using ct_master is working, I notice that sometimes
the tests fail because I cannot depend on the batches being executed in
sequence.  ct_master executes the test cases in parallel on the two nodes.
I could continue down this path by adding synchronization somehow, but I
feel that I am not really using ct_master in the way it was intended.

Has anyone on the list had experience writing an automated test suite where
the SUT is a single app running on a cluster of nodes?  Were you able to do
this with common_test, or did you end up rolling your own framework?

Thanks!

-Garret Smith
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20111219/e08e725d/attachment.htm>