How do I make TeamCity break build n if metric x is lower that it was in build n-1

So I've got TeamCity humming along quite nicely.  We (in a .NET shop) run tests, coverage, fxcop, ndepend (though only the default CQLs so far) during each build.

However, it's a mostly legacy code-base.  So test-coverage is low-to-nothing (though growing).  FxCop floods the inspections report with legions of terror and distastefulness.  NDepend violations are a crushing horde.

Some projects are in better shape than others; we have a couple greenfield bits that we're keeping well tended.  However, the legacy stuff presents a problem - don't want to turn on a draconian set of "Thou shalt ... <always use an IFormatProvider>" rules, since we'd grind to a halt making the build green again, and we'd be making changes that aren't covered by test-automation (and we have really gah-no-never-do-that-ever! things like hard-coding threads to particular locales and then expecting servers to be installed a particular way), and so we'd grind even more to a halt as we wrote test-automation for a system that's badly documented at best.

So, pragmatically, about the best _simplest_ thing we can do at at the moment is the keep-people-honest option, which is to fail the build if a checkin is detected that makes the code-base _worse_.  Worse, as defined simplistically, like:
* Test-coverage goes down (blanket coverage; more interesting things like coverage within particular namespaces can wait)
* FxCop warnings increase
* FxCop errors increase
* NDepend CQL violations of rule x increase
* <Insert other broad-brush static analysis metric here> gets worse

How do I get TeamCity to do that for us?  Obviously, not averse to tinkering with our build-script to do this.

One way that occurs to me is to get the build-script to publish said broad-brush metrics to a file and publish that at the end of a build (successful build or not), then as the last task of the build, pull the artifact from <.lastSuccessful>/artifacts (or whatever the URL would be), and do the arithmetic there.

Has anyone done this?  How did you do it?  Does it come out of the box?

2 comments
Comment actions Permalink

This is something we already think about. There is related issue in our tracker: http://youtrack.jetbrains.net/issue/TW-2539

0
Comment actions Permalink

Thanks - I've +1'd it.

0

Please sign in to leave a comment.