Build fails if at least one test fails despite configuration stating otherwise

Good evening,

I'm currently using TeamCity 5.1.4 (build 13550) with Visual Studio 2010 (10.0.30319.1), SVN (1.6 IIRC) and NUnit 2.5.5.

One of the goals I would like to achieve is to be able to do pre-tested commits, but with two very different scenarios:

1) Commit new tests, as well as methods or classes skeletons. The tests should fail but the build should work. This will ensure the tests were correctly written.

==> In this situation, I want the pre-tested commit to succeed (and therefore become an actual commit) IFF the build succeeds, that is, if there are no compilation errors.

2) Commit implementation that match tests and do not bring regressions. All tests should succeed.

==> in this situation, I want the pre-tested commit to succeed IFF there are no new failing tests, that is, if all tests which used to succeed, still succeed.

The runner used for this project is: Microsoft Visual Studio solution (.sln) runner.

"Fail build if" is set to :

[X] build process exit code is not zero

[  ] at least one test failed
[  ] an error message is logged by build runner
[  ] it runs longer than  minutes

[X] an out of memory or crash is detected (Java only)

Therefore, I'm expecting that when pre-testing commits from Visual Studio (using TeamCity add in), the "commit if" option will help me achieve the aforementioned desired effects in the following fashion:

- Scenario 1 ==> "build is successful"

- Scenario 2 ==> "no new tests failed"

It does not seem to work that way.

I'm now wondering whether there is something I'm not understanding or if this is a limitation to TeamCity or to the VS SLN runner.

Would somebody please enlight me?

Thanks for your time and consideration.

Best Regards,

Benjamin Neau

3 comments

Good evening,

It seems I can achieve somehow what I wanted to do by adding the tests first and using "no new tests failed" then adding the implementation using "build is successful".

This is getting better, but still it means there is no way to distinguish whether the tests are correctly written (read syntaxically correct) before actually implementating something to fix the tests.

Is there any plan to add a behavior such as the one I described?

Thanks for your time and consideration

Best Regards,

Benjamin Neau

0

There is no ability to make TeamCity commit a pre-tested commit if the build is failed and there are new failed tests in the build. Also, there are no plans so far to add such behavior.

It seems you need two build configuraitons one checking for compilation errors and another checking for failed tests.
Then you can use the compile one in your Scenario 1 and testing one in your Scenario 2.

0

Thanks for your answer Yegor.

We will do with the current settings.

I believe Jetbrains' approach has gone through much more thinking than mine therefore I see no reason not to use it.

Best Regards,

Benjamin Neau

0

Please sign in to leave a comment.