I'm currently using TeamCity 5.1.4 (build 13550) with Visual Studio 2010 (10.0.30319.1), SVN (1.6 IIRC) and NUnit 2.5.5.
One of the goals I would like to achieve is to be able to do pre-tested commits, but with two very different scenarios:
1) Commit new tests, as well as methods or classes skeletons. The tests should fail but the build should work. This will ensure the tests were correctly written.
==> In this situation, I want the pre-tested commit to succeed (and therefore become an actual commit) IFF the build succeeds, that is, if there are no compilation errors.
2) Commit implementation that match tests and do not bring regressions. All tests should succeed.
==> in this situation, I want the pre-tested commit to succeed IFF there are no new failing tests, that is, if all tests which used to succeed, still succeed.
The runner used for this project is: Microsoft Visual Studio solution (.sln) runner.
"Fail build if" is set to :
[X] build process exit code is not zero
[ ] at least one test failed
[ ] an error message is logged by build runner
[ ] it runs longer than minutes
[X] an out of memory or crash is detected (Java only)
Therefore, I'm expecting that when pre-testing commits from Visual Studio (using TeamCity add in), the "commit if" option will help me achieve the aforementioned desired effects in the following fashion:
- Scenario 1 ==> "build is successful"
- Scenario 2 ==> "no new tests failed"
It does not seem to work that way.
I'm now wondering whether there is something I'm not understanding or if this is a limitation to TeamCity or to the VS SLN runner.
Would somebody please enlight me?
Thanks for your time and consideration.