How's your Performance in TC 3.0? Ours... not so good

We're seeing serious performance degradation after migrating to TeamCity 3.0. Our server's CPU is maxed out at 100% and admin on the server as well as the web interface become unusable. We see a significant difference when we run with a couple of build configurations versus the whole lot we need to migrate from TC 2.0 -- approximately 200 build conf's. Anyone out there with this number of builds configured running 3.0? What hardware do you have allocated for your server? How's performance?

- Rene

4 comments

I'm just curious, but is your TeamCity web server acting as the agent too? That is do you have all these builds on a single box or are you using multiple agents?

Bob

0

No. We have a dedicated Windows 2003 server that we run the server piece on. With 6 windows build agents and 2 Linux ones that run on separate hosts (through VMWare). The only other thing our Build server is running besides Tomcat/TC is MySql.

Message was edited by:
Rene Medellin

0

Hello,

Please try to make several thread dumps on the server side when you
experience server slowdown and send these thread dumps to us.
You can make thread dumps with help of this tool:
http://www.adaptj.com/main/download

--
Pavel Sher
Software Developer
JetBrains, Inc.
http://www.jetbrains.com
"Develop with pleasure!"



"Rene Medellin" <medellre@yahoo.com> wrote in message
news:8109801.1200415021643.JavaMail.itn@is.intellij.net...

No. We have a dedicated Windows 2003 server that we run the server piece
on. With 6 windows build agents and 2 Linux ones. The only other thing our
Build server is running besides Tomcat/TC is MySql.



0

So after a lot of good feedback from the support team (Thanks Yegor!) we've identified the issue. Our biggest performance hit was the result of migrating 200+ build configurations each with their own VCS root. The way the migration works, it translates each 2.0 VCS root into a unique 3.0 VCS root. And the way VCS roots work in 3.0 you end up polling your VCS system for updates for each of those roots. Something that in 2.0 was not quite the same as it is in 3.0. So you really have to be careful about triggering duplicate lookups for the same root -- which is what was consuming all our CPU resources.
So we've identified 9 common VCS roots that contain all of our projects and we will set these up as shared VCS roots. From there we can control the triggering of individual project builds using the new project checkout rule functionality.
Upon initial testing of this approach we're back to more normal CPU utilization levels.

0

Please sign in to leave a comment.