How to improve build performance?

Hello,

We are experiencing some weird performance problems when attempting to build our projects thorugh TeamCity. Running the build on my local PC takes approximately 3 minutes (XP Pro SP2, 2GB RAM, Core 2 Duo @ 2.4GHz), however it's taking in excess of 16 minutes on the TeamCity Server - specs below:

- TeamCity Professional Version 3.1.2 (build 6881)
- Windows Server 2003 R2 Std, SP2
- Dual 3GHz Xeon Processors
- 2GB RAM

The server is a VM instance. I doubled the specs (was 1GB and 1 CPU) and ran the build again, however I saw no noticeable difference in the build time.

Is there anything else I can do to speed up the builds? I guess we could go to 4GB and 4 CPUs, but a bit reluctant considering that even though we have already doubled the specs of the server and haven't noticed any difference.

Any suggestions would be appreciated!

Thanks :)

7 comments
Comment actions Permalink

Hello,

Build duration depends not only on server performance (I guess you're running a build agent on the same PC as server),
but also on SCM server performance, because TeamCity collects build changes at the beginning of the build.

You may try to analyze build logs on build agent (see TeamCity/buildAgent/logs/teamcity-agent.log) to understand which
steps of your build take the most noticeable time.

Please also take in to account, that the whole build output is sent via network from build agent to server, and this adds some
overhead, so you can speedup your builds by reducing amount of the output your build produces (for instance, if your tests
write a lot of data to stdout).

Hope, this helps,
KIR

0
Comment actions Permalink

Thanks for your response.

I checked out the teamcity-agent.log file, and the build only generates about 40KB of log information (stopped the TC agent, cleared out the contents of the file, started the agent, ran the build). The parts of the build that take the most amount of time to run generate no output at all (the NAnt file specifies build.verbosity=minimal), so I'm assuming that probably isn't the source of the problem...

However, looking at the logs generated by the TeamCity Server (accessible by clicking on Build Logs > All Messages), it's about 21MB in size. Could this be the source of the problem? Is there a simple way to reduce the amount of output generated by TeamCity?

Thanks :)

0
Comment actions Permalink

Just a bit more info - I tried installing the same version of TeamCity on my Dev PC and it takes about 16 minutes to build, however, I also installed TeamCity 4 and it took approximately 4 minutes - huge improvement! The Build log was also only about 430KB.

0
Comment actions Permalink

Hello,

I really think you should stick to TeamCity 4 than. Anyway, we're going to release it in this year :)

Kind regards,
KIR

0
Comment actions Permalink

Could you please compare build logs from 3.1.2 and 4.0. What are the changes?
We have an issue on NAnt verbosity at http://www.jetbrains.net/tracker/issue/TW-4708
Thanks!

0
Comment actions Permalink

Thanks for your responses.

I have uploaded the 2 different log files from TC3 and TC4 to the issue you mentioned above. Hope it helps.

Upgrading to TC4 sounds great - should be great when it's released.

Thanks again :)

0
Comment actions Permalink

Also, are you running any NUnit tests. Pay attention to how much time you're spending on the following stages (depending on what build runner you're using):

  • Checking for changes (connection to source code control would be the bottleneck here)

  • Actual compilation (raw host machine resources, how your own projects build step is configured)

  • Unit tests (see which NUnitLauncher is getting executed here. Stock 3.1.1 TeamCity uses NUnit 2.1. which can be a bit slow. Download the patch for NUnit 2.4.x)

0

Please sign in to leave a comment.