Override test results
Completed
Hi!
I wonder if there is any way to override test result (from failed to passed) during running build? We send test results by service messages instead of xml report.
This is necessary for our rerun scheme (we run each test several times and want teamcity to show only the last result without making the build is red if the last test result was passed)
Thanks.
Please sign in to leave a comment.
Each test you register with testStarted/testFinished combination would count as a separate test execution. As per my quick check with this script:
echo ##teamcity[testStarted name='test1']
echo ##teamcity[testFailed name='test1' message='failure message']
echo ##teamcity[testFinished name='test1']
echo ##teamcity[testStarted name='test1']
echo ##teamcity[testFinished name='test1']
The test was registered twice with the build log (note the "run 2 times"):
As an alternative, I could suggest to amend the setup so that either:
a) service messages are printed out only when a final result of the test is available, or
b) at build configuration Failure Conditions, "at least one test failed" condition is checked off and another condition - "fail build on specific text in build log" - is added; for instance, you could print out in case any test had no green runs at all, even after rerun.
Just a quick follow-up: there is a feature request to allow to mark a test as successful if there are multiple executions of this test and at least one is green (https://youtrack.jetbrains.com/issue/TW-48674).