[erlang-bugs] Common Test 1.6.1 - Surefire

[Jóvan Sean Dippenaar]

Snippet of "skipped" test cases change for bamboo integration.


to_xml(#testcase{ group = Group, classname = CL, log = L, name = N, time = T, timestamp = TS, failure = F}) ->
    ["<testcase ",
     [["group=\"",Group,"\""]||Group /= ""]," "
     "name=\"",N,"\" "
     "time=\"",T,"\" "
     "timestamp=\"",TS,"\" "
     "log=\"",L,"\">",
     case F of
		 passed ->
		     [];
		 {skipped,Reason} ->
		     ["<failure type=\"skip\" message=\"Test ",N," in ",CL,
		      " skipped!\">", sanitize(Reason),"</failure>"];
		 {fail,Reason} ->
		     ["<failure message=\"Test ",N," in ",CL," failed!\" type=\"crash\">",
		      sanitize(Reason),"</failure>"]
     end,
     "</testcase>"];



/Jovan Sean Dippenaar - Aeonmind Enterprises



On 08 May 2012, at 10:49 AM, Jóvan Sean Dippenaar wrote:

> Good day
> 
> 
> I must have changed it when debugging "skipped" tests problem, I was experiencing in bamboo.
> 
> Due to observing that the "on_tc_skip" uses " _Tc " ( note small 'c' ), noting that Tc is an atom and TC
> is a record. Hence when debugging "on_tc_fail" I unwittingly removed the underscore for a debug, thinking that it would not conflict. ( the curse of long working hours :) )
> 
> 
> A note on skipped test cases in bamboo :
> 
> The JUnit parser in bamboo does not acknowledge "skipped" test cases, which unfortunately cause a "false positive" sistuation. 
> Changing the "skipped" to "failed", the bamboo junit parser shows the failed test cases. I personally prefer this when for project management reasons.
> 
> I hope is helps anyone integrating into bamboo.
> 
> 
> 
> / Jovan Sean Dippenaar - Aeonmind Enterprises 
> 
> 
> 
> <PastedGraphic-7.pdf>
> 
> On 07 May 2012, at 5:25 PM, Lukas Larsson wrote:
> 
>> Hello,
>> 
>> Thanks for the bug report and provided fix. I'll include it in the next release. 
>> 
>> I'm a bit uncertain what you mean with the first issue though. The code is already the way you propose it to be: https://github.com/erlang/otp/blob/bb1734e95a5f6a7315819c24bc1fdd799534c787/lib/common_test/src/cth_surefire.erl#L86
>> 
>> Lukas
>> 
>> On 05/05/12 14:01, Jóvan Sean Dippenaar wrote:
>>> 
>>> Errors in Common Test 1.6.1 - cth_surefire.erl
>>> 
>>> 
>>> cth_surefire.erl has a two errors :
>>> 
>>> 1. " on_tc_fail( TC, Res, State)" should be "on_tc_fail(_Tc, Res, State)", as the TC causes a bad match on inside the function with regards to the TC record. 
>>>     this causes surefire results to generate .xml which indicate passed tests when in actual fact the tests have failed.
>>> 
>>> 2. failures in "init_per_suite" cause "on_tc_skip" to be called which is correct up until "on_tc_skip" is called for "end_per_suite" as well. This also means that 
>>>     "post_end_per_suite" is never called, which is the problem. As "post_end_per_suite" takes all test cases results and places in State#state{ test_suites ).
>>>     This is important as the xml generation on terminate iterates thru the test suites.
>>> 
>>>     here's a fix :
>>> 
>>> on_tc_skip(Tc, Res, State) ->
>>>     TCs = State#state.test_cases,
>>>     [TC | RestTCs ] = TCs,
>>>     {CurTC, CurRestTCs} = case TC#testcase.failure of
>>> 						    	passed -> {TC, RestTCs};
>>> 						    	_OtherFailure -> 
>>> 						    		Name = atom_to_list(Tc),
>>> 						    		NTC = new_tc_rec(Name, State#state{timer = now()}),
>>> 						    		{NTC, TCs}
>>> 						    end,
>>>     
>>>     NewTC = CurTC#testcase{ 
>>>     						failure = {skipped, lists:flatten(io_lib:format("~p",[Res])) } 
>>> 						   },
>>>     NewState = State#state{ test_cases = [NewTC | CurRestTCs]},
>>>      
>>>     case Tc of
>>>     	end_per_suite ->
>>>     		 Suite = get_suite(NewState, TCs),
>>> 		    AltState =  NewState#state{ test_cases = [],  test_suites = [Suite | NewState#state.test_suites]},
>>>     		AltState;
>>>     	_Other ->
>>>     		NewState
>>>     end.
>>> 
>>> 
>>> 
>>> 
>>> / Jovan Sean Dippenaar - Aeonmind Enterprises
>>> 
>>> 
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> erlang-bugs mailing list
>>> erlang-bugs@REDACTED
>>> http://erlang.org/mailman/listinfo/erlang-bugs
>> _______________________________________________
>> erlang-bugs mailing list
>> erlang-bugs@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-bugs
> 
> _______________________________________________
> erlang-bugs mailing list
> erlang-bugs@REDACTED
> http://erlang.org/mailman/listinfo/erlang-bugs

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-bugs/attachments/20120508/8a98a3c6/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PastedGraphic-7.pdf
Type: application/pdf
Size: 35462 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-bugs/attachments/20120508/8a98a3c6/attachment.pdf>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-bugs/attachments/20120508/8a98a3c6/attachment-0001.htm>