What is the best way to conditionally mute a test?
TeamCity supports the ability to mute test failures but it seems to be unconditional. I have tests which fail with some error but non-deterministically. I'd still like to catch other errors but filter out only certain ones, which I can catch by examining the output. I also have some failures which aren't a function of a specific test but which can occur on any test also non-deterministically. Does TeamCity support muting these kinds of failures?
Please sign in to leave a comment.
Uri,
There is no direct support for such test muting.
Can you please describe this in more detail?
How/where do you want to analyze test failure? Inside the build? Inside a TeamCity plugin? In some external process and mark the tests muted "post-factum"?
Do you want to implement your own analyzing logic or want some predefined bundled methods (like regexp matching)?
Ideally, TeamCity would support this feature natively. I'm thinking that a new box would be added to the mute dialog for a closure condition, which gets the test output as a parameter along with other information about the test (XML node perhaps so that you can get the name, time run, etc). I have it currently implemented in Groovy by reading in a separate file. It would also be useful to add a new menu item to manage these closures where one can be defined which isn't specific to a given test.
Uri,
If you want to handle the parsing programmatically, you can write TeamCity plugin to handle the task. GroovyPlug is an example of possible Groovy events processing in TeamCity.
I believe it can be extended to mute a test in a running build based on some logic.
However, muting a test is not yet part of open API (so the API can change in future versions without extra notice).
Thanks for the suggestion