How does TeamCity count tests?
Hello,
I'm integrating tests written in rspec (ruby). We are running them using the jruby-maven-plugin (http://github.com/mkristian/jruby-maven-plugins).
Unfortunately the tests aren't counted by TeamCity into the total - which is seriously morale-depriving ;)
I was wondering what file do I have to generate so that the tests are counted? So far I tried creating a target/surefire-reports/TEST-Ruby.xml file with the following content:
<?xml version="1.0" encoding="UTF-8" ?>
<testsuite failures="0" time="1.339" errors="0" skipped="0" tests="31" name="Ruby">
but no effect.
Adam
Please sign in to leave a comment.
Please provide more details on your configuration.
Did you configure import from XML reports?
Do you mean settings in Runner: Maven2 > XML Report Processing ? I already have FindBugs configured there and it seems I can configure only one :/. But I was hoping to mimick whatever surefire/testng outputs that make the tests being counted.
Adam
You can send service message to the ouput with a path to junit-like or surefire-like xml reports, read more on this here: http://confluence.jetbrains.net/display/TW/XML+Test+Reporting
Some nice info there, thanks :)
I output a line using the antrun plugin:
##teamcity[importData type='surefire' path='${project.build.directory}/surefire-reports/TEST-Ruby.xml
when the file already exists, but the tests don't get counted (xml-report-plugin is in TC by default so I didn't do anything special to install it); although publishing the test results html as an artifact worked, so it's always a step forward.
Do you maybe know which file exactly is "importable" and what does it need to contain? I can generate either a testng-like or juint-like report, I'd just need to know the "template" :)
Adam
JUnit like reports should work. You need to specify junit as a report type in this case.
If they won't work, please attach xml reports and build log, so that we could reproduce the problem.
Still nothing unfortunately. In the build log I can see:
[17:03:26]: Importing data from '/home/user/tools/teamcity/work/f9f7e13c9243f205/myproject/target/surefire-reports/TEST-Ruby.xml' with 'junit' processor
[17:03:26]: [Ant JUnit report watcher] /home/user/tools/teamcity/work/f9f7e13c9243f205/myproject/target/surefire-reports/TEST-Ruby.xml
This is after all the tests are run and the file already exists.
The TEST-Ruby.xml file contains only:
<testsuite failures="0" time="2.002" errors="0" skipped="0" tests="48" name="Ruby">
</testsuite>
Adam
Well, the file must have details on each test: their names and statuses. It is not enough to have total number of tests.
I modified the plugin to include the testcases, and now it works perfectly - thanks a lot for the help! :)
Adam
I am also having similar problem. Could you please explain what changes you made with the plugin?