[erlang-questions] Common Test last test report

[Loïc Hoguin]

On 04/20/2017 01:03 PM, Peter Andersson E wrote:
> On 02/25/2017 12:20 AM, Loïc Hoguin wrote:
>> Hello,
>>
>> It's still difficult to get to the last test report in Common Test.
>> Currently the URI looks like this:
>>
>> file:///home/essen/ninenines/cowboy/logs/ct_run.ct_cowboy@localhost.2017-02-25_00.02.28/ninenines.cowboy.rfc7540_SUITE.logs/run.2017-02-25_00.02.29/suite.log.html
>>
>>
>> Which is great, I can always go back and compare, and sometimes that's
>> what I want (when I test across Erlang releases, for example). But
>> when I'm fixing a bug, I tend to run a small set of tests, maybe a
>> single suite, or sometimes even just a single group or a single test
>> case. When I do that, things becomes more difficult than they should
>> be because I can't simply refresh the page to get the new report, I
>> have to go back, refresh on the all_runs.html page, and follow the
>> links from there.
>>
>> Would it be possible for Common Test to maintain a separate directory
>> always containing the latest report? For example:
>>
>> file:///home/essen/ninenines/cowboy/logs/ct_run.ct_cowboy@localhost.latest/ninenines.cowboy.rfc7540_SUITE.logs/run.latest/suite.log.html
>>
>>
>> With a URI like this I can repeat the test run, refresh in my browser
>> and get the new report instantly. (And keep the history separate
>> should I need to compare.)
>>
>> Alternatively, have an option to disable the timestamps so I can just
>> refresh when I need it, although the history wouldn't be preserved in
>> this case.
>>
>> Cheers,
>>
>
> Hi Loïc + anyone else interested,
>
> I've made a solution to this problem as follows:
>
> In the top level log directory (i.e. where the top level index files,
> all_runs.html and index.html, are stored) an html file,
> suite.log.latest.html, is written at the start of each test run, which
> contains an redirection (immediate, without a click) to the latest saved
> suite.log.html (test case overview). There is a link to this redirection
> page in the footer of suite.log.html. This makes it possible to jump
> immediately to the latest test results from any test case overview page
> without having to navigate to the top level index first. So this is
> useful when you're running the same tests repeatedly and want to update
> and/or compare results.
>
> The shortcoming of this design, however, is that it ignores the test
> name. It's the latest report in 'logdir' that is jumped to, not the
> latest report for the current set of tests (like e.g. what you get in
> the top level index.html page). Example: If you run a set of tests, A,
> then repeat this set, A', then run a new set of tests, B, then if you're
> analyzing the results of A and click the latest results link, you're
> redirected to the report for B, not A'.
>
> The workaround in the above example is either to use different top level
> log directories for A and B test runs, or to navigate from
> suite.log.html for A to the top level index.html (via the link in the
> footer) and go to A' from there.
>
> My question is, what do you guys prefer? Is the current implementation
> as I descibe it sufficient and useful, or would a more complex solution
> with "name scoped redirections" make more sense? I'd appreciate the input!

Hello!

Sorry I didn't reply earlier.

The solution you describe would work well enough for me I think. It 
helps with "run tests, check the results, fix things, run tests again, 
compare the results and see what broke/got fixed". I do not have much 
value for keeping the logs around personally, so would not need the 
workaround you mention and/or a better fix.

Maybe someone else would.

Thanks for your great work!

-- 
Loïc Hoguin
https://ninenines.eu