[erlang-questions] Common Test last test report

[Peter Andersson E]

On 02/25/2017 12:20 AM, Loïc Hoguin wrote:
> Hello,
>
> It's still difficult to get to the last test report in Common Test. 
> Currently the URI looks like this:
>
> file:///home/essen/ninenines/cowboy/logs/ct_run.ct_cowboy@localhost.2017-02-25_00.02.28/ninenines.cowboy.rfc7540_SUITE.logs/run.2017-02-25_00.02.29/suite.log.html 
>
>
> Which is great, I can always go back and compare, and sometimes that's 
> what I want (when I test across Erlang releases, for example). But 
> when I'm fixing a bug, I tend to run a small set of tests, maybe a 
> single suite, or sometimes even just a single group or a single test 
> case. When I do that, things becomes more difficult than they should 
> be because I can't simply refresh the page to get the new report, I 
> have to go back, refresh on the all_runs.html page, and follow the 
> links from there.
>
> Would it be possible for Common Test to maintain a separate directory 
> always containing the latest report? For example:
>
> file:///home/essen/ninenines/cowboy/logs/ct_run.ct_cowboy@localhost.latest/ninenines.cowboy.rfc7540_SUITE.logs/run.latest/suite.log.html 
>
>
> With a URI like this I can repeat the test run, refresh in my browser 
> and get the new report instantly. (And keep the history separate 
> should I need to compare.)
>
> Alternatively, have an option to disable the timestamps so I can just 
> refresh when I need it, although the history wouldn't be preserved in 
> this case.
>
> Cheers,
>

Hi Loïc + anyone else interested,

I've made a solution to this problem as follows:

In the top level log directory (i.e. where the top level index files, 
all_runs.html and index.html, are stored) an html file, 
suite.log.latest.html, is written at the start of each test run, which 
contains an redirection (immediate, without a click) to the latest saved 
suite.log.html (test case overview). There is a link to this redirection 
page in the footer of suite.log.html. This makes it possible to jump 
immediately to the latest test results from any test case overview page 
without having to navigate to the top level index first. So this is 
useful when you're running the same tests repeatedly and want to update 
and/or compare results.

The shortcoming of this design, however, is that it ignores the test 
name. It's the latest report in 'logdir' that is jumped to, not the 
latest report for the current set of tests (like e.g. what you get in 
the top level index.html page). Example: If you run a set of tests, A, 
then repeat this set, A', then run a new set of tests, B, then if you're 
analyzing the results of A and click the latest results link, you're 
redirected to the report for B, not A'.

The workaround in the above example is either to use different top level 
log directories for A and B test runs, or to navigate from 
suite.log.html for A to the top level index.html (via the link in the 
footer) and go to A' from there.

My question is, what do you guys prefer? Is the current implementation 
as I descibe it sufficient and useful, or would a more complex solution 
with "name scoped redirections" make more sense? I'd appreciate the input!

Best,
Peter

Ericsson AB, Erlang/OTP