[erlang-questions] Running distributed tests with common test

[Peter Andersson]

[Answered this yesterday, but appearently the mail didn't get through,
so I try again...]

Hi Fredrik,

Unfortunately, the current ct_master implementation requires that all
test host nodes have a shared file system (for the links from the
master_runs page to be valid anyway). A custom event handler won't help
much since it's mainly statistics and error messages that the event
handler can receive and report anyway. Common Test doesn't send general
io such as printouts from the SUT and the test cases, the CT framework
log, and other debug info to the event manager, and I assume you want to
be able to read all that info on the master node, right?

What you could do is maybe copy the log files from the test host nodes
to the master node during the test run, by means of a CT hook. The links
in master_runs.html will then still be invalid of course, but this we
could implement a fix for - like introduce a new/modified test
specification term that tells ct_master which path to use for the links
in the master log file. The links between files inside the log file
structure on each test host are relative, so that shouldn't cause any
problems.

I'm sure a solution like this is not as simple as it may sound, but I
don't mind playing around a bit with this myself to see what works and
what doesn't and how we can improve the current ct_master implementation
in this regard. In the future, we'd like to make the io handling part of
Common Test open for other formats than html (by means of plugins) and
that should also make it much easier to implement a distributed test
mode where io can be passed from host nodes to master node. That's a big
job though, and not one we've had time to start yet.

Best regards,
Peter

Ericsson AB, Erlang/OTP


2012/3/6, Fredrik Svahn <fredrik.svahn@REDACTED>:
> Hi,
>
> I have a setup where I am using ct_master:run_test/2 to run tests on target
> nodes which each have a local file system.
>
> On the server I get the following html result pages:
> master_runs.html
> ct_master_run.2012-03-06_14.17.16/index.html
>
> The links on the latter page are however broken, since they refer to files
> on each target's local file system. In the Users Guide it is stated: "The
> CT Master server writes progress information to HTML log files similarly to
> the regular CT server. The logs contain test statistics and links to the
> log files written by each independent CT server" which seems to support
> that what I am seeing is indeed the intended behaviour.
>
> Nevertheless, since I know CT is used for a lot of embedded testing I am
> wondering if there is a way of having the test targets report back *all*
> results and logs to the master server (as opposed to writing the results
> locally)? Would writing a custom event handler help?
>
> BR /Fredrik
>
> PS. this is the full command I use to start the tests, just in case I have
> missed some simple config option:
> ct_master:run_test(target@REDACTED, [{suite,[array_SUITE]},{auto_compile,
> false}, {basic_html, true}, batch]).
>