Ant JUnit report watcher reporting incorrect test durations

Hi, Scott,

Could you, please, update xml-report-plugin and make sure that this problem still occurs. If it really occurs, please, attach full build log.

Hi,

I updated to the latest version. Restarted the server.

Also made sure that agent and server times were in sync.

The problem still exists.

Attached is the full log and the xml report.

Thanks,

Scott

Thanks a lot for the attachments.

One more thing we need to localize the problem: could you, please, turn agent log to debug mode (go to <build agent installation path>\conf\teamcity-agent-log4j.xml and remove lines that are marked with "DELETE THIS LINE FOR ENABLING DEBUG LOGGING" text), run the build and provide here the agent log file (is at <build agent installation path>\logs\teamcity-agent.log). Agent restart before running build is not necessary.

Hi.

Attached is the agent log...

Another observation...If a test failed, the time is correct. otherwise, the first test is the only correct time reported.

Scott

Hi,

Any progress on this issue?

Also, I've attached another XML report which has failures with stacktraces. The stacktraces do not show up in TC.

Thanks,

Scott

Thanks for information and attachment!

This problems occur since AntUnit produces xml reports with structure that differs from Ant junit task xml reports schema. It will be fixed in the nearest time by supporting this new format.

Alternatively, is there a fix I could do to XMLAntUnitListener.java in AntUnit?

I see that the actual ANT builds here -- http://teamcity.jetbrains.com/viewType.html?tab=buildTypeStatusDiv&buildTypeId=bt133 -- use AntUnit and their test results show up correctly.

Thanks,

Scott

The originally supported format was:

<!ELEMENT testsuites (testsuite*)>

<!ELEMENT testsuite (properties, testcase*,
                  failure?, error?,
                   system-out?, system-err?)>
<!ATTLIST testsuite name      CDATA #REQUIRED>
<!ATTLIST testsuite tests     CDATA #REQUIRED>
<!ATTLIST testsuite failures  CDATA #REQUIRED>
<!ATTLIST testsuite errors    CDATA #REQUIRED>
<!ATTLIST testsuite time      CDATA #REQUIRED>
<!ATTLIST testsuite package   CDATA #IMPLIED>
<!ATTLIST testsuite id        CDATA #IMPLIED>


<!ELEMENT properties (property*)>

<!ELEMENT property EMPTY>
<!ATTLIST property name  CDATA #REQUIRED>
<!ATTLIST property value CDATA #REQUIRED>

<!ELEMENT testcase (failure?, error?, system-out?, system-err?)>
<!ATTLIST testcase name       CDATA #REQUIRED>
<!ATTLIST testcase classname  CDATA #IMPLIED>
<!ATTLIST testcase time       CDATA #REQUIRED>

<!ELEMENT failure (#PCDATA)>
<!ATTLIST failure message CDATA #IMPLIED>
<!ATTLIST failure type    CDATA #REQUIRED>

<!ELEMENT error (#PCDATA)>
<!ATTLIST error message CDATA #IMPLIED>
<!ATTLIST error type    CDATA #REQUIRED>

<!ELEMENT system-err (#PCDATA)>

<!ELEMENT system-out (#PCDATA)>

AntUnit reports were supposed to fit it, but actually in many cases they don't.
But still I can't get any AntUnit xml repots schema at public resources.

One more source of possible report data skipping is the "parsing on the fly".
May be you can observe how do reports appear on disk during the build? Are they published once at the end of all test case or partly after each test is finished?

One more source of possible report data skipping is the "parsing on the fly".

>> I can try to disable parsing via the build config setup page.

May be you can observe how do reports appear on disk during the build? Are they published once at the end of all test case or partly after each test is finished?

Some good news!

- By disabling real-time parsing of XML reports AND enabling post test parsing via service messages, I now get the correct test times.

The bad news:

- Tests with errors do not show stacktraces...

I've attached another XML report.

Thanks for your help!

Scott

Hi Victory,

Any news on this problem? Do you have any beta code you want me to try?

Thanks,
Scott

Hi,

I'm getting another error too.

D:\Apps\BuildAgent\work\6.4smoke\xml-test-reports\TEST-simple-smoke-tests_xml.xml: failed to parse with Ant JUnit parser

The file is attached.

Thanks

Scott

There is a FIX for error stacktraces loss, please upload - http://teamcity.jetbrains.com/repository/download/bt143/.lastSuccessful/xml-report-plugin.zip

Hi, Scott,

Excuse me for my silence.

TEST-simple-smoke-tests_xml.xml seems to be well-stuctured. But this error tells that at the end of the build the report is still unfinished or wrong-formated.
Do you get it regularly?

May be you can attach any snapshots of a report as it is at some stage of testing (after several tests finish)?

I uploaded the zip. do I need to restart the server?

I re-ran it and it does not show the stacktrace still, nor the exception message.

thanks,

Scott

To install the plugin put xml-report-plugin.zip into <TeamCity Data Directory>/plugins folder and restart the server.
You should also see that after the server restart agents start upgrading.

>> I'm getting another error too.

>> D:\Apps\BuildAgent\work\6.4smoke\xml-test-reports\TEST-simple-smoke-tests_xml.xml: failed to parse with Ant JUnit parser

Please look attentively at the TEST-simple-smoke-tests_xml.xml (I attach it again for the convenience ) - there is no closing tag for last <testcase> element. Error is reported due to wrong XML structure of the report.

Hi.
I still cannot see my stacktraces or any error messages even though my tests have failed.
Any ideas?
Thanks,
Scott

Hi, Scott.

Thanks for attention to this problem!

It's a bit strange that you still can not see error stacktraces, these bug was fixed.
I'l try to investigate your problem once more.

What TeamCity version are you using now?
Are you sure plugin version is up-to-date? (Suggest you http://teamcity.jetbrains.com/viewLog.html?guest=1&tab=artifacts&buildTypeId=bt143&buildId=lastSuccessful if you use 4.5.5)
Did you notice my previous post about TEST-simple-smoke-tests_xml.xml?
Besides, are test durations shown correctly?

Hi,

I'm using TC 4.5.4. And yes, I took the latest build from your TC instance. I've restarted the server since I updated the plugin. Is there a way to confirm the version of the plugin that is running?

My durations are correct now. I no longer have my runner configured to watch for the xml file. Instead I only issue an importData service message once all tests are done.

I've re-attached my latest xml file for you to look at.

Thanks,

Scott

Thanks for attachments!

To make sure that new plugin version is loaded please examine your teamcity-server.log (at <TeamCity home\log>) for warning like this:

Plugin xml-report-plugin[version: ... path: ...\xml-report-plugin] will not be loaded because it is overridden by plugin: xml-report-plugin[version: ... path: ...\xml-report-plugin]

It should make clear what plugin version is working.
Copy this warning here, please.
It looks as if you have old one.