[erlang-questions] Automated testing of a distributed application

[Garret Smith]

Thank you Lukas!

That was the realization I was coming to about common_test that you said
perfectly: it's for distributing tests, not testing distributed
applications.

Looks like I'll be writing the infrastructure myself.  At least I have the
slave and rpc modules to help me out.

-Garret


On Tue, Dec 20, 2011 at 4:47 AM, Lukas Larsson
<lukas@REDACTED>wrote:

> Hello!
>
> The ct_master module is not really meant to test distributed
> applications, but rather to distribute your tests. It helps you when
> you have 1000 tests to run, but can only finish 500 per day and you
> need to run all of them each day.
>
> When testing distributed application with common_test, I use the
> ct_slave (or just even slave) module and configure slaves to be
> started on node X and Y and then run test suites on node Z which
> trigger test behavior either through rpc or SIP och whatever interface
> you want to test. Serializing test calls and making sure that no test
> cases interfere with each other I do in the testcase and suite as I
> would with a non-distributed application.
>
> Lukas
>
> On Tue, Dec 20, 2011 at 1:56 AM, Garret Smith <garret.smith@REDACTED>
> wrote:
> > I just finished setting up an automated test suite for my distributed
> > application using the common_test "Large Scale Testing" support.  It was
> > rough, but *almost* works.  It starts up 2 local nodes, starts my app on
> > both nodes, runs a few test cases from my suite on node1, a second batch
> of
> > cases on node2, then yet another batch of cases on node1.
> >
> > The test cases are designed to work by contacting the distributed app on
> > node1, modifying it's state through the API, then contacting the app on
> > node2 to verify that the state changes are reflected there, then
> repeating
> > from the perspective of node2.
> >
> > sample test spec, invoked with ct_master:run/1
> >
> > {logdir, "../logs"}.
> >
> > {alias, testdir, "."}.
> >
> > {init, [n1, n2], {node_start, [{erl_flags, "-pa ../../ebin"},
> > {monitor_master, true}]}}.
> >
> > {merge_tests, false}.
> >
> > {cases, n1, testdir, distributed_SUITE, [create_table, insert_1]}.
> > {cases, n2, testdir, distributed_SUITE, [read_1, update_2]}.
> > {cases, n1, testdir, distributed_SUITE, [read_2]}.
> >
> >
> >
> > Now that my basic setup using ct_master is working, I notice that
> sometimes
> > the tests fail because I cannot depend on the batches being executed in
> > sequence.  ct_master executes the test cases in parallel on the two
> nodes.
> > I could continue down this path by adding synchronization somehow, but I
> > feel that I am not really using ct_master in the way it was intended.
> >
> > Has anyone on the list had experience writing an automated test suite
> where
> > the SUT is a single app running on a cluster of nodes?  Were you able to
> do
> > this with common_test, or did you end up rolling your own framework?
> >
> > Thanks!
> >
> > -Garret Smith
> >
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://erlang.org/mailman/listinfo/erlang-questions
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20111220/def34f46/attachment.htm>