[erlang-questions] Automated testing of a distributed application

[Lukas Larsson]

The reason why there is no specific test support for distributed apps is
partly because ct is mainly used to do black box tests of embedded
applications which need a completely different and much much bigger
architecture than you normal Erlang application.

Also if you have a lot of test suites I would recommend writing a ct hook
which starts and monitors slaves throughout the tests and maybe restart the
applications when a test fails.

There are a lot of great things which you can do with ct if you just take
the time to dig below the surface. Let me know if you have any other
questions or issues!

Lukas
On Dec 20, 2011 7:04 PM, "Garret Smith" <garret.smith@REDACTED> wrote:

> Thank you Lukas!
>
> That was the realization I was coming to about common_test that you said
> perfectly: it's for distributing tests, not testing distributed
> applications.
>
> Looks like I'll be writing the infrastructure myself.  At least I have the
> slave and rpc modules to help me out.
>
> -Garret
>
>
> On Tue, Dec 20, 2011 at 4:47 AM, Lukas Larsson <lukas@REDACTED
> > wrote:
>
>> Hello!
>>
>> The ct_master module is not really meant to test distributed
>> applications, but rather to distribute your tests. It helps you when
>> you have 1000 tests to run, but can only finish 500 per day and you
>> need to run all of them each day.
>>
>> When testing distributed application with common_test, I use the
>> ct_slave (or just even slave) module and configure slaves to be
>> started on node X and Y and then run test suites on node Z which
>> trigger test behavior either through rpc or SIP och whatever interface
>> you want to test. Serializing test calls and making sure that no test
>> cases interfere with each other I do in the testcase and suite as I
>> would with a non-distributed application.
>>
>> Lukas
>>
>> On Tue, Dec 20, 2011 at 1:56 AM, Garret Smith <garret.smith@REDACTED>
>> wrote:
>> > I just finished setting up an automated test suite for my distributed
>> > application using the common_test "Large Scale Testing" support.  It was
>> > rough, but *almost* works.  It starts up 2 local nodes, starts my app on
>> > both nodes, runs a few test cases from my suite on node1, a second
>> batch of
>> > cases on node2, then yet another batch of cases on node1.
>> >
>> > The test cases are designed to work by contacting the distributed app on
>> > node1, modifying it's state through the API, then contacting the app on
>> > node2 to verify that the state changes are reflected there, then
>> repeating
>> > from the perspective of node2.
>> >
>> > sample test spec, invoked with ct_master:run/1
>> >
>> > {logdir, "../logs"}.
>> >
>> > {alias, testdir, "."}.
>> >
>> > {init, [n1, n2], {node_start, [{erl_flags, "-pa ../../ebin"},
>> > {monitor_master, true}]}}.
>> >
>> > {merge_tests, false}.
>> >
>> > {cases, n1, testdir, distributed_SUITE, [create_table, insert_1]}.
>> > {cases, n2, testdir, distributed_SUITE, [read_1, update_2]}.
>> > {cases, n1, testdir, distributed_SUITE, [read_2]}.
>> >
>> >
>> >
>> > Now that my basic setup using ct_master is working, I notice that
>> sometimes
>> > the tests fail because I cannot depend on the batches being executed in
>> > sequence.  ct_master executes the test cases in parallel on the two
>> nodes.
>> > I could continue down this path by adding synchronization somehow, but I
>> > feel that I am not really using ct_master in the way it was intended.
>> >
>> > Has anyone on the list had experience writing an automated test suite
>> where
>> > the SUT is a single app running on a cluster of nodes?  Were you able
>> to do
>> > this with common_test, or did you end up rolling your own framework?
>> >
>> > Thanks!
>> >
>> > -Garret Smith
>> >
>> > _______________________________________________
>> > erlang-questions mailing list
>> > erlang-questions@REDACTED
>> > http://erlang.org/mailman/listinfo/erlang-questions
>> >
>>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20111220/01c647bd/attachment.htm>