What does the JUnit XML format used by TeamCity look like?
In particular, I have a custom test runner that generates JUnit compatible XML files and would like to see how TeamCity determines if a test was ignored and for what reason. Failures and errors are straightforward.
Please sign in to leave a comment.
Hello, Uri,
TeamCity provides several bundled runners that automatically report tests.
In case of custom runners tests may be not reported automatically, but if tool supports generating compatible XML report TeamCity can import tests from this report.
To import such reports from custom runners you need to use TeamCity "XML report processing" build feature.
It's Ant JUnit parser actually supports reports matching the following DTD (retrieved from Apache Ant src).
As you can see it doesn't provide any information about tests ignoring.
So tools that generate Ant JUnit-like reports have freedom to save ignored tests in different ways.
Currently we support following structures to mark test as ignored (different tools normally use one of them):
1) ...<testcase executed="false"...>...
2) ...<testcase...> <skipped/></testcase>...
3) ...<testcase status="ignored"...>...
No ignoring reason is supported.
BTW, what test runner do you use?
The DTD is:
<!ELEMENT testsuites (testsuite*)>
<!ELEMENT testsuite (properties, testcase*,
failure?, error?,
system-out?, system-err?)>
<!ATTLIST testsuite name CDATA #REQUIRED>
<!ATTLIST testsuite tests CDATA #REQUIRED>
<!ATTLIST testsuite failures CDATA #REQUIRED>
<!ATTLIST testsuite errors CDATA #REQUIRED>
<!ATTLIST testsuite time CDATA #REQUIRED>
<!ATTLIST testsuite package CDATA #IMPLIED>
<!ATTLIST testsuite id CDATA #IMPLIED>
<!ELEMENT properties (property*)>
<!ELEMENT property EMPTY>
<!ATTLIST property name CDATA #REQUIRED>
<!ATTLIST property value CDATA #REQUIRED>
<!ELEMENT testcase (failure?, error?, system-out?, system-err?)>
<!ATTLIST testcase name CDATA #REQUIRED>
<!ATTLIST testcase classname CDATA #IMPLIED>
<!ATTLIST testcase time CDATA #REQUIRED>
<!ELEMENT failure (#PCDATA)>
<!ATTLIST failure message CDATA #IMPLIED>
<!ATTLIST failure type CDATA #REQUIRED>
<!ELEMENT error (#PCDATA)>
<!ATTLIST error message CDATA #IMPLIED>
<!ATTLIST error type CDATA #REQUIRED>
<!ELEMENT system-err (#PCDATA)>
<!ELEMENT system-out (#PCDATA)>
I discovered this page on build messages, which I think is what I'm looking for. It is too bad that it doesn't show messages. Your suggestion also helps, though it doesn't match the provided DTD. Will test it later.
Our tests are submitted over OGE from a priopriety script. We are running system tests that take hours or days and need fine grained resource requirement specifications. The script dumps the XML file when it's done so that's why I needed to understand what the format looks like.
What do mean when saying "it doesn't show messages"?
Iimporting XML report with the help of ##teamcity[importData ...] service message actually tells TeamCity where to look for xml reports.
The reports (if type='junit') are expected to match the above mentioned DTD with modifications concerning ignored tests that I've described.
Do you have any schema for your reports? Or may be it's possible to have a look at any example?
That's referring to your initial reply, that tests are only showed as ignored but the reason is not included. That page mentions that tests can be marked as ignored through service messages, rather than a non-standard JUnit XML file.