stdout from surefire/failsafe tests missing in "Show stacktrace"

We are running a Maven 3.0.3 build of Java code within TeamCity 6.0.3 using surefire/failsafe for unit and integration tests respectively. Some time in the past 4 weeks the stdout from tests stopped being available in TeamCity, and we're not sure where.

We use redirectTestOutputToFile in surefire/failsafe and I can see the "-output.txt" files in target/surefire-reports for the build agent, so this part is working fine. However when I click "Show stacktrace" for a failed test it only shows me the exception, not the stdout and stderr from the test.

We used to get

EXCEPTION STACKTRACE
------- Stdout: -------
@HL
@HL-------------------------------------------------------
@HL T E S T S
@HL-------------------------------------------------------
@SLRunning com.blah.blah.NameOfTest
STDOUT


This used to work.

Does anyone have any idea how I could debug this further? I don't see any errors in TeamCity log files, but it's very frustrating as it makes test failures difficult-to-impossible to debug for integration tests. In those last few weeks a couple of things were changed - upgraded to 6.0.3 from 6.0.1 and bumped version of surefire plugin to 2.8.1 in order to address a separate bug there but not sure what could have caused this to change.

6 comments
Comment actions Permalink

Chad,

Sorry for the delay in replying.

Can you please examine surefire test report XMLs to see if the test output in there?
TeamCity parses the XMLs to get the test run information.

0
Comment actions Permalink

No, there is no stdout/stderr inside the XML surefire reports. Should it really be there though - not sure I've ever seen the STDOUT come in that XML - would it be in a CDATA section or something?

0
Comment actions Permalink
Hi Chad,
There are elements <system-out> and <system-err> in the surefire/junit report schema, which come alongside the <failure> element.
TeamCity has always been using surefire xml reports as the only source of test data in Maven builds, which means if you saw stdout before, then it definitely was taken from the XMLs.
There is simply no other way of extracting this information from Maven builds.
Probably not all surefire versions can put it into the reports. Check you haven't changed the surfire plugin version in your pom since the last time you saw the stdout in teamcity.
0
Comment actions Permalink

I doubt that's exactly the same issue given it's 3 years ago - this was working perfectly well up until 6 weeks or so ago. Pretty sure nothing related to logging libraries or logging configs has changed - and if I remove redirectTestOutputToFile, it logs into regular Maven log as expected. Which surefire versions does Jetbrains test with?

I'll try rolling back surefire versions, but I'm not convinced. I looked at another build by another team on a different TC instance who don't use redirectTestOutputToFile at all and TeamCity correctly seems to be able to correctly extract the logs from the Maven stream somehow and report them with stacktraces (I've seen that there's nothing in their .xml surefire reports either, nor any -output.txt files) - so I guess that TeamCity behaves differently when redirect is on to off? I'll play a bit more I guess in either case.

0
Comment actions Permalink

OK, I think I'm convinced now :)

Part of my confusion was that I was expecting to see <std-out> in the surefire reports for successes; didn't realise it should only be there for failures. So the issue seems to be that they've broken the surefire plugin in 2.8 and 2.8.1. My guess is that this is caused by something similar to http://jira.codehaus.org/browse/SUREFIRE-725 - possibly logging into the xml report relies on it going through the logger rather than a system.out?

Unfortunately we'd upgraded to this version to address issues like http://jira.codehaus.org/browse/SUREFIRE-704 which used to plague our build :(

On the plus side, I just tried with 2.8.2-SNAPSHOT and it seems the behaviour has been restored.

0

Please sign in to leave a comment.