stdout from surefire/failsafe tests missing in "Show stacktrace"
We are running a Maven 3.0.3 build of Java code within TeamCity 6.0.3 using surefire/failsafe for unit and integration tests respectively. Some time in the past 4 weeks the stdout from tests stopped being available in TeamCity, and we're not sure where.
We use redirectTestOutputToFile in surefire/failsafe and I can see the "-output.txt" files in target/surefire-reports for the build agent, so this part is working fine. However when I click "Show stacktrace" for a failed test it only shows me the exception, not the stdout and stderr from the test.
We used to get
EXCEPTION STACKTRACE
------- Stdout: -------
@HL
@HL-------------------------------------------------------
@HL T E S T S
@HL-------------------------------------------------------
@SLRunning com.blah.blah.NameOfTest
STDOUT
This used to work.
Does anyone have any idea how I could debug this further? I don't see any errors in TeamCity log files, but it's very frustrating as it makes test failures difficult-to-impossible to debug for integration tests. In those last few weeks a couple of things were changed - upgraded to 6.0.3 from 6.0.1 and bumped version of surefire plugin to 2.8.1 in order to address a separate bug there but not sure what could have caused this to change.
Please sign in to leave a comment.
Chad,
Sorry for the delay in replying.
Can you please examine surefire test report XMLs to see if the test output in there?
TeamCity parses the XMLs to get the test run information.
No, there is no stdout/stderr inside the XML surefire reports. Should it really be there though - not sure I've ever seen the STDOUT come in that XML - would it be in a CDATA section or something?
BTW: You doesn't seem to be the only person who have ever had this problem
http://maven.40175.n5.nabble.com/Getting-useful-logging-from-surefire-and-testng-td98993.html
I doubt that's exactly the same issue given it's 3 years ago - this was working perfectly well up until 6 weeks or so ago. Pretty sure nothing related to logging libraries or logging configs has changed - and if I remove redirectTestOutputToFile, it logs into regular Maven log as expected. Which surefire versions does Jetbrains test with?
I'll try rolling back surefire versions, but I'm not convinced. I looked at another build by another team on a different TC instance who don't use redirectTestOutputToFile at all and TeamCity correctly seems to be able to correctly extract the logs from the Maven stream somehow and report them with stacktraces (I've seen that there's nothing in their .xml surefire reports either, nor any -output.txt files) - so I guess that TeamCity behaves differently when redirect is on to off? I'll play a bit more I guess in either case.
OK, I think I'm convinced now :)
Part of my confusion was that I was expecting to see <std-out> in the surefire reports for successes; didn't realise it should only be there for failures. So the issue seems to be that they've broken the surefire plugin in 2.8 and 2.8.1. My guess is that this is caused by something similar to http://jira.codehaus.org/browse/SUREFIRE-725 - possibly logging into the xml report relies on it going through the logger rather than a system.out?
Unfortunately we'd upgraded to this version to address issues like http://jira.codehaus.org/browse/SUREFIRE-704 which used to plague our build :(
On the plus side, I just tried with 2.8.2-SNAPSHOT and it seems the behaviour has been restored.