JUnit Ant - new test failure reporting

How does TeamCity determine if a test failure is new in a particular build?

There are a number of tests that we currently expect to fail and they fail in every build, however, TeamCity always reports them as new failures:
Tests failed: 34 (34 new), passed: 592

BuildRunner: Ant
TestFramework: JUnit

Ant target:
]]>
<include name="*/eep/common//Test.java"/>
<exclude name="*/Abstract.java"/>
<exclude name="*/BaseTest.java"/>
]]>
<include name="*/eep/mrpt//Test.java"/>
<exclude name="*/Abstract.java"/>
<exclude name="*/BaseTest.java"/>
]]>
<include name="*/eep/pitc//Test.java"/>
<exclude name="*/Abstract.java"/>
<exclude name="*/BaseTest.java"/>
]]>
<include name="*/eep/tip//Test.java"/>
<exclude name="*/Abstract.java"/>
<exclude name="*/BaseTest.java"/>
]]>
<include name="*/eep/afm//Test.java"/>
<exclude name="*/Abstract.java"/>
<exclude name="*/BaseTest.java"/>
</fileset>
</batchtest>
</junit>
</target>

Is there something I need to do to ensure that these tests are recognized as the same tests from build to build?

2 comments

Hi Tim,

In fact, the only thing which is required from tests to be recognized as repeatable failures - same name. Please make sure that tests have the same name in consequent builds.

Please also note that in TeamCity 3.0 there was a bug when test status was cached and cache was not cleaned when build finishes. The bug was fixed in 3.1.

If the problem persists, please try to prepare a small case for reproducing the problem and file bug report at http://www.jetbrains.net/tracker/

Regards,
KIR

0

Kirill,

Yes the test names are exactly the same in subsequent builds. So are the assertions, exceptions, and stack traces.

We are using TeamCity Professional Version 3.1 (build 6760) on a Windows Server 2003 box.

I will see what I can do about producing a small test case, but I may not have time to do that for a while.

Thanks,

Tim

0

Please sign in to leave a comment.