Fitnesse integration with TC

I have a custom version of fitnesse that provides the service messages for ongoing tests following the docs

Now fitnesse tests are reported with the junit tests which a great first step.
It is so great that TC will now track how fast how individual tests run over time (great for performance regression... almost need an alert that shows the tests that slow down beyond a threshold or sort the list based on the amount of degradation). Even better is the fact that we know get immediate feedback when a test fails. Before that we had to wait up to 15mn for the build to finish! So thank for such a great product! ]]>

So now the next step is to differentiate junit tests from fitnesse tests. How could I do the following:

  1. 2 Tests tabs one for junit tests and one for fitnesse tests with descriptive names .

  2. Modify the build status from "5430 tests passed" to "5090 junit tests passed, 349 fit test passed" as the build runs

  3. Modify the build status once the build is complete to "6000/0 junit, 400/0 fit, 81% coverage, 2405 classes, 254KLOC"


Then I will look into the graphs to replicate the sadely fake pictures that you have in the docs... :)

Thanks in advance

Jacques

Edited by: Jacques Morel on Jul 18, 2008 1:14 AM

Edited by: Jacques Morel on Jul 18, 2008 2:19 AM

5 comments

I was playing with the status messages. The documentation does not explain what is the difference between FAILURE, SUCCESS, NORMAL, ERROR especially in what ways it impact the display in different views.

Also I noticed that even though I have a custom status that is indeed used in the Project view and in the Build Overview once the build is finished, in the Build Overview page while the build is running, the status message below the summary rectangle is the standard one that only include junit tests. Shouldn't it be the custom message?

To explain better what my previous second point was I will present some example of what it does today and what I would ideally want:

Currently

Ideally

Issue

Tests passed: 5394; Fitnesse passed: 337

Unit tests: 5057; Func tests: 337

Right now 'Tests passed' is the total junit+fitnesse. I would like to differentiate. I also would like to get that number but have freedom to change the format

Tests failed: 1 (1 new), passed: 5393; Fitnesse failed: 1, passed: 336

Unit tests: 5057; Func tests: 1 failed, 5392 passed

Same as previously, there is no value and only noise at displaying the same failures in both categories

0

It would be great if somebody from JetBrains would comment on my suggestion. Thanks

0

Hello Jacques,

Regarding FAILURE, SUCCESS, NORMAL, ERROR. By the moment FAILURE==ERROR and SUCCESS==NORMAL and there is no other difference.
We might add some difference between ERROR and FAILURE status later ( to distinguish internal errors and build failures), but only after Calcutta.

Regarding build status text. On the server side you can write a plugin which creates an arbitrary status text. You can access general statistics about test count
from our API (SBuild.getShortStatistics() or SBuild.getFullStatistics()).

But so far, TeamCity cannot distinguish different types of tests (unit, functional, etc.).

As a workaround I suggest using the fact that in Calcutta the name of the test suite is included into the full test name. With this information, you can try to
detect functional tests, count them and create full status text via RunningBuildImpl::setBuildDescription call (most likely, this method will go to OpenAPI).

We might differentiate tests from different suites in the future, but this task is out of the Calcutta scope.

Kind regards,
KIR

0

Is there documentation on how to build this server plugin and do you have a open source example I can base my plugin from?

Thanks

Jacques

0

Please sign in to leave a comment.