[erlang-bugs] Common Test 1.6.1 - Surefire

[Jóvan Sean Dippenaar]

Good day


I must have changed it when debugging "skipped" tests problem, I was experiencing in bamboo.

Due to observing that the "on_tc_skip" uses " _Tc " ( note small 'c' ), noting that Tc is an atom and TC
is a record. Hence when debugging "on_tc_fail" I unwittingly removed the underscore for a debug, thinking that it would not conflict. ( the curse of long working hours :) )


A note on skipped test cases in bamboo :

The JUnit parser in bamboo does not acknowledge "skipped" test cases, which unfortunately cause a "false positive" sistuation. 
Changing the "skipped" to "failed", the bamboo junit parser shows the failed test cases. I personally prefer this when for project management reasons.

I hope is helps anyone integrating into bamboo.



/ Jovan Sean Dippenaar - Aeonmind Enterprises 





On 07 May 2012, at 5:25 PM, Lukas Larsson wrote:

> Hello,
> 
> Thanks for the bug report and provided fix. I'll include it in the next release. 
> 
> I'm a bit uncertain what you mean with the first issue though. The code is already the way you propose it to be: https://github.com/erlang/otp/blob/bb1734e95a5f6a7315819c24bc1fdd799534c787/lib/common_test/src/cth_surefire.erl#L86
> 
> Lukas
> 
> On 05/05/12 14:01, Jóvan Sean Dippenaar wrote:
>> 
>> Errors in Common Test 1.6.1 - cth_surefire.erl
>> 
>> 
>> cth_surefire.erl has a two errors :
>> 
>> 1. " on_tc_fail( TC, Res, State)" should be "on_tc_fail(_Tc, Res, State)", as the TC causes a bad match on inside the function with regards to the TC record. 
>>     this causes surefire results to generate .xml which indicate passed tests when in actual fact the tests have failed.
>> 
>> 2. failures in "init_per_suite" cause "on_tc_skip" to be called which is correct up until "on_tc_skip" is called for "end_per_suite" as well. This also means that 
>>     "post_end_per_suite" is never called, which is the problem. As "post_end_per_suite" takes all test cases results and places in State#state{ test_suites ).
>>     This is important as the xml generation on terminate iterates thru the test suites.
>> 
>>     here's a fix :
>> 
>> on_tc_skip(Tc, Res, State) ->
>>     TCs = State#state.test_cases,
>>     [TC | RestTCs ] = TCs,
>>     {CurTC, CurRestTCs} = case TC#testcase.failure of
>> 						    	passed -> {TC, RestTCs};
>> 						    	_OtherFailure -> 
>> 						    		Name = atom_to_list(Tc),
>> 						    		NTC = new_tc_rec(Name, State#state{timer = now()}),
>> 						    		{NTC, TCs}
>> 						    end,
>>     
>>     NewTC = CurTC#testcase{ 
>>     						failure = {skipped, lists:flatten(io_lib:format("~p",[Res])) } 
>> 						   },
>>     NewState = State#state{ test_cases = [NewTC | CurRestTCs]},
>>      
>>     case Tc of
>>     	end_per_suite ->
>>     		 Suite = get_suite(NewState, TCs),
>> 		    AltState =  NewState#state{ test_cases = [],  test_suites = [Suite | NewState#state.test_suites]},
>>     		AltState;
>>     	_Other ->
>>     		NewState
>>     end.
>> 
>> 
>> 
>> 
>> / Jovan Sean Dippenaar - Aeonmind Enterprises
>> 
>> 
>> 
>> 
>> 
>> 
>> _______________________________________________
>> erlang-bugs mailing list
>> erlang-bugs@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-bugs
> _______________________________________________
> erlang-bugs mailing list
> erlang-bugs@REDACTED
> http://erlang.org/mailman/listinfo/erlang-bugs

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-bugs/attachments/20120508/04c108b5/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PastedGraphic-7.pdf
Type: application/pdf
Size: 35462 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-bugs/attachments/20120508/04c108b5/attachment.pdf>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-bugs/attachments/20120508/04c108b5/attachment-0001.htm>