Run recently failed tests first?

I use the Nant runner to build a C# solution and run the nunit unit test. and teamcity have a very nice feature "Reduce test failure feedback time:                       Run recently failed tests first".

For example. I have ten test projects. and I use script to run it one by one. and  one test in test project 3  fails .

If I turn on this feature.
For the first time. it runs test project 1,2 and fails on test project 3 ( the 6th test).
For the second time. it runs test project 1,2 and run the 6th test first and fail.

it means it run recently failed tests first on the test project 3.

Is there a way that it run recently failed tests first on all test projects? What I mean is it should run 6th test in test project 3  first before it run test project 1,2.

If I run multi  nunit projects. does it have the same behavior  as the dll file? Does  nunit project support this feature?

Thanks

3 comments
Comment actions Permalink

What way do you run NUnit tests? Could you please attach a part of your build script.

TeamCity applies test reordering inside any of nunit runner calls. It's impossible to reorder those calls in your build script.

0
Comment actions Permalink

I see.
becaseu I need to run nunit project. So the nunit-console.exe is my only choice. I use 'foreach" to run nunit project one by one.

It means that teamcity have no way to apply test reordering. right?

Teamcity can apply test reordering only if multi assemblies locate in the same <nunit2> task.
For example.
                    <nunit2>
                        <formatter type="Plain" />
                        <test assemblyname="bin/nunit/assemblyName1.dll"  />
                        <test assemblyname="bin/nunit/assemblyName2.dll"  />
                    </nunit2>

If I use multi <nunit2> task or run the <nunit2> task one by one.teamcity have no way to apply test reordering
                    <nunit2>
                        <formatter type="Plain" />
                       <test assemblyname="bin/nunit/assemblyName1.dll"  />
                    </nunit2>
                    <nunit2>
                        <formatter type="Plain" />
                         <test assemblyname="bin/nunit/assemblyName2.dll"  />
                    </nunit2>
or
            <foreach item="Folder" in="bin/tests" property="foldername" verbose="true" >
                <do>
                <property name="assemblyName" value="${path::get-file-name(foldername)}" />
                    <nunit2>
                        <formatter type="Plain" />
                        <test assemblyname="bin/nunit/${assemblyName}.dll"  />
                    </nunit2>
                </do>
            </foreach>


Is that correct?

0
Comment actions Permalink

NUnit-Console.exe + TeamCity Addin for NUnit does not supports tests reordeing.

Test reordering will work only if you use either TeamCity embedded NUnit runner or <nunit2> task in NAnt.

You need to run all test assemblies within one call to test runner to make it reorder assemblies sequence.

0

Please sign in to leave a comment.