My company currently uses Emma to generate statistics around unit test coverage. Currently we have so many lines of code that any individual checkin doesn't change the overall unit coverage stats. What I am looking for is the ability to cross reference the checkins diffs (lines added, updated, deleted), with the Emma line coverage stats to generate a version control commit statistic that displays: Unit Test Coverage % for added lines, Unit Test Coverage % for updated lines (obviously deleted lines won't be covered).
I think that the Unit Coverage % for newly added / updated lines is the most valuable metric for a development team. This simple metric will drive the correct behavior, which is to only commit unit tested code. If you update code and it wasn't previously tested, you should add unit tests before you commit the updates. Of course, I'd want to add historical reporting, and per user statistics later so you could have a leader board that showed the top 10 commiters to a project by unit test coverage.
So now for my question. Is there anything like this that currently exists for TeamCity or any other open source build server? If no solution currently exists, how hard would it be to write a TeamCity plug that after a build, looks at both the Emma output and the version control lines added/updated/deleted for each file and calculates the metrics that I listed above? I'm happy to write this plugin as I feel it would be helpful not only for our company but everyone following TDD (test driven development) or just seeking better commit level unit testing metrics. I'd like to make this an open source plugin because I strongly believe that these new metrics will help improve the overall quality of TDD and open source projects in general.
Again, developing this plugin is well within my skill set (7+ years Java development with extensive third party plugin experience), however from the TeamCity plugin docs, I don't know where to begin. So if this doesn't exist could someone tell me in TeamCity terms, what type of plugin this would be? How I can access both Emma and VC changes for a given build? Being a developer, I don't need extensive documentation, just pointing me at a couple open source plugins that are similar or have pieces that I need would be helpful. Basically, you write the words and I'll write the code.
Please help make the world of Unit Testing a better place!
Thanks in advance.