Unit Test Step (Ant) Hangs on New Server, but Not Old One
We've recently upgraded from TeamCity 8.1.1 to TeamCity 2019.1.1. The transition is complete and all builds are running and building successfully. However, we are working on speeding up builds now and we've noticed a difference between the logs running on the old and new server (we have triggers on both that run the same builds on each server simultaneously).
Two projects in particular are having issues. Though the new server is usually faster on most build steps, it is significantly slower when it hits the Run Unit Tests (Ant) step. There is a fairly consistent 7-8 minute difference between the servers on this one step.
What can cause this significant difference? Would it be potentially TC side or should we consider taking a look at changing tests?
Please sign in to leave a comment.
Hi,
thanks for your comment. I'm afraid that with this little information it's almost impossible for us to tell. Build logs are well structured and contain very detailed information and, critically for performance checking, timestamps. Ant is quite a verbose runner, so you should be able to see whether the slowness is throughout the run, or comes to specific points. If it comes to specific parts of it, then you can easily see where to look for issues. If on the other hand the issue is general to that build step, then it might make sense to try to understand whether there are any differences between what each step is running, whether the actual build agents have any difference in their configuration (typical here is third party tools interacting with the machine), whether caches have been built on the previous installation that need to be built now in the new ones, etc. Also obviously, 7-8 minutes without knowing the scale of the build says quite little about it. In a 10 minute build, that is obviously a lot and should indicate an actual issue. In a 6 hours build, it might just be variance, or that some small tweaks on the runner might have made it a tad slower. I assume it's the former, otherwise you wouldn't be reporting, but in order to investigate deeper it would be useful to understand the scale.
If you keep observing this behavior, another set of steps which might help is available here: https://www.jetbrains.com/help/teamcity/common-problems.html#CommonProblems-BuildworkslocallybutfailsormisbehavesinTeamCity