All my tests have errors (not failures) but the build is a success

We have just had a successful build after all the tests had an error (we are using Ant and JUnit 4). The test classes could not be initialised due to a coding error, so the tests did not 'fail' but had 'errors'.

Our build configuration is set to fail the build if 'build process exit code is not zero' or 'at least one test failed'. Neither of these conditions catch the 'all tests have errors' situation.

Possible solutions I've thought of are:
- set a non-zero exit code for test errors - how would I do that?
- have TeamCity detect the errors and fail the build itself - is that possible?
- set the 'haltonerror' attribute in the junit ant task to 'true' - would this cause the build to fail?

I've attached our build log.

Many thanks,

Comment actions Permalink

I think you nailed it

haltonerror - Stop the build process if an error occurs during the test run.

~ Orn

Comment actions Permalink

Please submit a bug report with this build log attached.

Pavel Sher

Comment actions Permalink

Thanks, I tried that and it does correctly trap the problem


Please sign in to leave a comment.