How to incorporate third-party product reports (like jdepend, cobertura)

First, let me say that I have been really impressed by the product so far. The combination of build grid, remote build, notifications options and user friendly UI makes it a very attractive offering! Keep up the good work.

Since it is such a nice tool... I am in the process to migrate our project from cruisecontrol. We have made some CC extensions in order to see the reports of different tools we use to measure quality:
1. fitnesse
2. jdepend
3. checkstyle
4. cobertura
...

These tools may be broken down into
1. single file reports and
2. multi-files, iframe based reports.

It is easy to integrate the single file reports through an artifact definition.
It isn't for the multi-file kind.

So here to my question: what would be the best way to integrate them?

I can see 2 ways:
1. a simple way : no visual integration, just a link to the report index page. The agent somehow copies all report files
2. a comprehensive way : same as 1. but TC shows report display in another tab like emma does I believe

I would like to get 1. running as soon as possible.
I would like to get to 2. eventually. Is emma/coverage plugin source available? Where can I find information about hosting the project in jetbrains subversion/jira/wiki?

Thanks in advance

Jacques

11 comments
Comment actions Permalink

Hello Jacques,

Regarding the first way. In fact, you can specify a whole directory with your report files in artifacts path, so the whole directory can be specified as artifact and will be browseable from TeamCity pages.

The second way involves development of TeamCity server-side plugin. For emma reports, we upload an archive with all html reports and access it's content as a whole.

The sources of server-side part of EMMA coverage plugin I've just attached to
http://www.jetbrains.net/confluence/display/TW/Plugin+Development page. This page contains general information on how to create plugins and deploy them with TeamCity. Sorry, it is rather incomplete but still useful for general overview.

Hope this helps and please feel free to ask questions,

KIR

0
Comment actions Permalink

Thanks for the sample. I think that will be a huge help for me.

--Tim

0
Comment actions Permalink

Thanks a lot for the source. This will help tremendously.
Unfortunately just putting the directory as an artifact does not work since all files inside it becomes available from the artifacts popup. Corbutera has one file per class! So this method can only work if we can "hide" a directory.

Looking at a tighter integration, I would think that in order to speed the creation of third-party report support you should create a generic "report inclusion" plugin: That plugin would take a generated report as an index.html linking to a bunch of detail pages in a subdirectory content/]]>.html and it would add a tab and display the report in it.
I might take a shot at it with your code so I can do jdepend and corbutera in one plugin. However this might require configuration of the plugin at the project level and would ideally be done from the admin console. Does the API support this today?

Jacques

BTW: I am just curious why JetBrains went for emma instead of corbutera. You guys usually pick the best in class third-parties but in this instance, I disagree. The fact that emma does not support branch coverage is a huge drawback of emma IMHO.

0
Comment actions Permalink

Tim,

I am curious. Like I put in my previous post, I will do jdepend and cobertura. What plugin will you implement?

Jacques

0
Comment actions Permalink

Looking at a tighter integration, I would think that in order to speed the creation of third-party report support you should create a generic "report inclusion" plugin: That plugin would take a generated report as an index.html linking to a bunch of detail pages in a subdirectory content/<detail page>.html and it would add a tab and display the report in it.

This would rule greatly. Even relatively prosaic build steps like javadoc and junitreport would benefit from this. There are so many things that this would work for, I just wonder if the tab rendering code is up to handling >20 tabs.

--Dave Griffith

0
Comment actions Permalink

Macker. Checkstyle.

They work well enough in the artifacts directory except that you only know to dig into them if there is a problem.

--Tim

0
Comment actions Permalink

I've created an issue that would enable 2) automatically for a large number of cases : http://www.jetbrains.net/jira/browse/TW-1665 . Please vote and comment. I feel this would be a very powerful way of making TeamCity extensible, and would probably be very easy to implement.

--Dave Griffith

0
Comment actions Permalink

Kirill,

Regarding http://www.jetbrains.net/jira/browse/TW-1665, sorry I missed your conclusion. Somehow the notification from JIRA did not reach my inbox (may a filter from my side...). Anyhow it is awesome to get this in the next build!

Back to business: some of these reports have summary data that would be great to display at the same level as the unit tests summary on the build list.
In my case, we are running fitnesse as our acceptance test frameworks and we would like to show like for unit tests, how many ran and how many failed.
So instead of the current
Tests passed: <% unit tests>
it could be
Tests passed: 1234 junit, 789 fitnesse

In the case of failure, it would be great to be able to tell what made the build fail: 10 checkstyle errors, 3 cycles...

Couldn't this be a simple xml build summary report that could be used by the build to annotate a build in TC? More than a summary report, this file would be the definition of a TC build. It would specify all customizable attributes of a build. Maybe that file could be in fact what tells TC what artifacts are available and where...

Let me know what you think. Should I open another issue?

Thanks again for this quick turn around time. As Dave, I am hooked. In addition if you are turning around this thing that quickly, this is enjoyable. When can we expect the new build?

Jacques

null

0
Comment actions Permalink

Definitely open an issue. Getting showing third party rollups in summaries (and notifications!) would indeed be very cool.

--Dave Griffith

0
Comment actions Permalink

Thanks again for this quick turn around time. As
Dave, I am hooked. In addition if you are turning
around this thing that quickly, this is enjoyable.
When can we expect the new build?


We'll try hard to bring out the next build in the middle of the next week.

Regards,
KIR

0

Please sign in to leave a comment.