[erlang-questions] Automated testing of a distributed application

[Lukas Larsson]

Here are some, there is also an example in the documentation for
hooks: http://erlang.org/doc/apps/common_test/ct_hooks_chapter.html#id77379

https://github.com/erlang/otp/blob/master/lib/common_test/src/cth_log_redirect.erl
https://github.com/erlang/otp/blob/master/lib/test_server/src/ts_install_cth.erl
https://github.com/garazdawi/cth_tools/tree/master/src

all modules names *_cth.erl in
https://github.com/erlang/otp/tree/master/lib/common_test/test/ct_hooks_SUITE_data/cth/tests

Lukas

On Tue, Dec 20, 2011 at 10:46 PM, Zabrane Mickael <zabrane3@REDACTED> wrote:
> Hi Lukas,
>
> Do you have any (working) code snippet to share about CT Hooks?
>
> Regards,
> Zabrane
>
> On Dec 20, 2011, at 10:30 PM, Lukas Larsson wrote:
>
> The reason why there is no specific test support for distributed apps is
> partly because ct is mainly used to do black box tests of embedded
> applications which need a completely different and much much bigger
> architecture than you normal Erlang application.
>
> Also if you have a lot of test suites I would recommend writing a ct hook
> which starts and monitors slaves throughout the tests and maybe restart the
> applications when a test fails.
>
> There are a lot of great things which you can do with ct if you just take
> the time to dig below the surface. Let me know if you have any other
> questions or issues!
>
> Lukas
>
> On Dec 20, 2011 7:04 PM, "Garret Smith" <garret.smith@REDACTED> wrote:
>>
>> Thank you Lukas!
>>
>> That was the realization I was coming to about common_test that you said
>> perfectly: it's for distributing tests, not testing distributed
>> applications.
>>
>> Looks like I'll be writing the infrastructure myself.  At least I have the
>> slave and rpc modules to help me out.
>>
>> -Garret
>>
>>
>> On Tue, Dec 20, 2011 at 4:47 AM, Lukas Larsson
>> <lukas@REDACTED> wrote:
>>>
>>> Hello!
>>>
>>> The ct_master module is not really meant to test distributed
>>> applications, but rather to distribute your tests. It helps you when
>>> you have 1000 tests to run, but can only finish 500 per day and you
>>> need to run all of them each day.
>>>
>>> When testing distributed application with common_test, I use the
>>> ct_slave (or just even slave) module and configure slaves to be
>>> started on node X and Y and then run test suites on node Z which
>>> trigger test behavior either through rpc or SIP och whatever interface
>>> you want to test. Serializing test calls and making sure that no test
>>> cases interfere with each other I do in the testcase and suite as I
>>> would with a non-distributed application.
>>>
>>> Lukas
>>>
>>> On Tue, Dec 20, 2011 at 1:56 AM, Garret Smith <garret.smith@REDACTED>
>>> wrote:
>>> > I just finished setting up an automated test suite for my distributed
>>> > application using the common_test "Large Scale Testing" support.  It
>>> > was
>>> > rough, but *almost* works.  It starts up 2 local nodes, starts my app
>>> > on
>>> > both nodes, runs a few test cases from my suite on node1, a second
>>> > batch of
>>> > cases on node2, then yet another batch of cases on node1.
>>> >
>>> > The test cases are designed to work by contacting the distributed app
>>> > on
>>> > node1, modifying it's state through the API, then contacting the app on
>>> > node2 to verify that the state changes are reflected there, then
>>> > repeating
>>> > from the perspective of node2.
>>> >
>>> > sample test spec, invoked with ct_master:run/1
>>> >
>>> > {logdir, "../logs"}.
>>> >
>>> > {alias, testdir, "."}.
>>> >
>>> > {init, [n1, n2], {node_start, [{erl_flags, "-pa ../../ebin"},
>>> > {monitor_master, true}]}}.
>>> >
>>> > {merge_tests, false}.
>>> >
>>> > {cases, n1, testdir, distributed_SUITE, [create_table, insert_1]}.
>>> > {cases, n2, testdir, distributed_SUITE, [read_1, update_2]}.
>>> > {cases, n1, testdir, distributed_SUITE, [read_2]}.
>>> >
>>> >
>>> >
>>> > Now that my basic setup using ct_master is working, I notice that
>>> > sometimes
>>> > the tests fail because I cannot depend on the batches being executed in
>>> > sequence.  ct_master executes the test cases in parallel on the two
>>> > nodes.
>>> > I could continue down this path by adding synchronization somehow, but
>>> > I
>>> > feel that I am not really using ct_master in the way it was intended.
>>> >
>>> > Has anyone on the list had experience writing an automated test suite
>>> > where
>>> > the SUT is a single app running on a cluster of nodes?  Were you able
>>> > to do
>>> > this with common_test, or did you end up rolling your own framework?
>>> >
>>> > Thanks!
>>> >
>>> > -Garret Smith
>>> >
>>> > _______________________________________________
>>> > erlang-questions mailing list
>>> > erlang-questions@REDACTED
>>> > http://erlang.org/mailman/listinfo/erlang-questions
>>> >
>>
>>
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>