JUnit Ant - new test failure reporting

How does TeamCity determine if a test failure is new in a particular build?

There are a number of tests that we currently expect to fail and they fail in every build, however, TeamCity always reports them as new failures:
Tests failed: 34 (34 new), passed: 592

BuildRunner: Ant
TestFramework: JUnit

Ant target:
]]>
<include name="*/eep/common//Test.java"/>
<exclude name="*/Abstract.java"/>
<exclude name="*/BaseTest.java"/>
]]>
<include name="*/eep/mrpt//Test.java"/>
<exclude name="*/Abstract.java"/>
<exclude name="*/BaseTest.java"/>
]]>
<include name="*/eep/pitc//Test.java"/>
<exclude name="*/Abstract.java"/>
<exclude name="*/BaseTest.java"/>
]]>
<include name="*/eep/tip//Test.java"/>
<exclude name="*/Abstract.java"/>
<exclude name="*/BaseTest.java"/>
]]>
<include name="*/eep/afm//Test.java"/>
<exclude name="*/Abstract.java"/>
<exclude name="*/BaseTest.java"/>
</fileset>
</batchtest>
</junit>
</target>

Is there something I need to do to ensure that these tests are recognized as the same tests from build to build?

2 comments
Comment actions Permalink

Hi Tim,

In fact, the only thing which is required from tests to be recognized as repeatable failures - same name. Please make sure that tests have the same name in consequent builds.

Please also note that in TeamCity 3.0 there was a bug when test status was cached and cache was not cleaned when build finishes. The bug was fixed in 3.1.

If the problem persists, please try to prepare a small case for reproducing the problem and file bug report at http://www.jetbrains.net/tracker/

Regards,
KIR

0
Comment actions Permalink

Kirill,

Yes the test names are exactly the same in subsequent builds. So are the assertions, exceptions, and stack traces.

We are using TeamCity Professional Version 3.1 (build 6760) on a Windows Server 2003 box.

I will see what I can do about producing a small test case, but I may not have time to do that for a while.

Thanks,

Tim

0

Please sign in to leave a comment.