Team City maven build is slow when having extensive console output logging in tests
Having a really simple maven project (https://github.com/inekrashevych/teamcity-test) with one test is ~22 times slower than running from console on local machine or running locally on build agent (redirect output to file)
All the test does is printing lines to console (which is simple use case with huge projects):
LongStream.range(0, 1_000_000).forEach(l ->
System.out.println("test " + l + " this is a long line test that is supposed to cause troubles at console. yes, it is super long, you can not read this even. It is even longer than reading the whole log from book"));
The result of the build is 212Mb log file.
TeamCity build takes >7min, while console build takes 19 sec on local machine (printing to console) and 17 sec (printing to file (having SSD)).
I've tried to enable Performace feature to build but that is not showing any insights.
Is there an open issue there? Is there a way for team city to redirect a log into file to speed up build? Why it is taking sooooo long?
Note: Using latest teamcity 2018.1, also teamcity and build agent are spread across the network.
Please sign in to leave a comment.
Hi,
I'm not really sure that it's a "simple use case with huge projects" to have millions of lines being posted to stdout without actual work being done in between. In our experience, tens of thousands is realistic in large projects, but more than that can signal that the project might need splitting in smaller parts, even if just to keep maintenance of the project doable. Obviously, a million of very long lines are going to take space, and given that you aren't actually doing CPU work between the printlns, but simply outputting, that means that TeamCity has a lot less time to "catch up" with the build (remember, all this lines are being sent between processes agent and server, often through the network). The server is being bombarded by the log messages, which is very likely not just creating performance issues in the agent but also in the server if it's not large enough.
The build log itself cannot be redirected to a file. If you want to redirect your build output to files, that's something you can do on your own build script. I can recommend doing that and testing whether teamcity actually is delayed by that much in that case. If it took the same time, that would very likely signal an issue.
This said, this comments would explain why the delays, massively long log files, etc. If your projects really do have this characteristics, I'd recommend opening an issue in our tracker https://youtrack.jetbrains.com/issues/TW and describing the details a lot more. If you are experiencing issues with your actual projects, please report them as such. If they contain private information, feel free to open the issues with private visibility, or use the "Submit a request" button on the top of this page.