TeamCity 6.5.1 (build 17834) - Snapshot Dependancies - slow builds?

We have a number of teamcity projects that have dependancies on each other when triggering certain builds.

The issues we are experiencing are that these builds seem to take forever to complete (ie. >5 mins per snapshot dependancy), it appears to be taking a lot of time checking for changes (from Visual Studio) for each dependancy (in the chain) and as a result, the agents (4 of them in total) remain Idle most of the time until the build are triggered.

Is this the normal chain of events for snapshot dependancies and does the time taken sound reasonable?

Are there ways to improve the overall speed of dependancy builds?

Any suggestions/comments much appreciated.

Thanks

3 comments
Comment actions Permalink

Chances are this is the same problem as: http://youtrack.jetbrains.net/issue/TW-17273
We'll release 6.5.2 version with this bug fixed soon, most likely tomorrow.

0
Comment actions Permalink

Hi,

Thank you for the advise.

We have upgraded TeamCity over the weekend to 6.5.2 (build 17935) but problem still exists.

Any other advise would be much appreciated.

Regards

Eric

 

Looking at the various logs it looks like the bottleneck is the retrieval of the source code from our two VCS systems, TFS and StartTeam, although StarTeam seems not to have as many issues (maybe because is used less).

 

Example #1: clean checkout takes nearly 20 minutes for some builds

 

(TFS)

[13:57:25]: Checking for changes

[14:06:40]: Clean build enabled: removing old files from C:\TeamCity\BuildAgent\work\cacc507cacde8532

[14:06:40]: Clearing temporary directory: C:\TeamCity\BuildAgent\temp\buildTmp

[14:06:40]: Checkout directory: C:\TeamCity\BuildAgent\work\cacc507cacde8532

[14:06:40]: Updating sources: server side checkout... (7m:04s)

[14:07:10]: [Updating sources: server side checkout...] Will perform clean checkout

[14:07:10]: [Updating sources: server side checkout...] Clean checkout reasons

[14:07:10]: [Clean checkout reasons] Checkout directory is empty or doesn't exist

[14:07:10]: [Clean checkout reasons] "Clean all files before build" turned on

[14:07:10]: [Updating sources: server side checkout...] Building clean patch for VCS root: Studio Main branch

[14:12:53]: [Updating sources: server side checkout...] Transferring cached clean patch for VCS root: Studio Main branch

[14:12:35]: [Updating sources: server side checkout...] Repository sources transferred: 150.83Mb total

[14:12:35]: [Updating sources: server side checkout...] Average transfer speed: 12.33Mb per second

[14:12:35]: [Updating sources: server side checkout...] Removing C:\TeamCity\BuildAgent\work\cacc507cacde8532

[14:12:35]: [Updating sources: server side checkout...] Updating C:\TeamCity\BuildAgent\work\cacc507cacde8532

[14:16:12]: Publishing internal artifacts (4s)

...

(StarTeam)

[09:44:50]: Checking for changes

[09:57:29]: Clearing temporary directory: C:\TeamCity\BuildAgent\temp\buildTmp

[09:57:29]: Checkout directory: C:\TeamCity\BuildAgent\work\961c495d5e66994c

[09:57:29]: Updating sources: server side checkout... (47s)

[09:57:29]: [Updating sources: server side checkout...] Building incremental patch for VCS root: Process Control Main branch

[09:58:15]: [Updating sources: server side checkout...] Repository sources transferred: 457.55Kb total

[09:58:16]: [Updating sources: server side checkout...] Updating C:\TeamCity\BuildAgent\work\961c495d5e66994c

[09:58:17]: Publishing internal artifacts (4s)

...

 

 

As you can see to do a clean checkout takes nearly 20 minutes in the first example, this is even before the build starts. Note also that that the “Checking for changes” is taking a long time in both TFS and StarTeam builds.

 

 

Example #2: Checking for changes in a Snapshot dependency chain can take over 20 minutes

 

From "teamcity-vcs.log"

 

[2011-07-11 06:22:16,897]   INFO [rmal executor 1] -      jetbrains.buildServer.VCS - Done requesting patch for root tfs: http://*masked out*/ {instance id=111, parent id=16} cleanPatch = true, prevVersion=null, newVersion=15582, buildId = 54052; took 636 sec 903 msec

0
Comment actions Permalink

Clean patch can indeed be slow, as it retrieves all of the sources from VCS. How much time does it take locally to checkout the whole source tree?
It is not clear why checking for changes takes so much time. Do you have teamcity-vcs.log for this period of time? For the time when "Checking for changes" was logged into the build log? If so, please send them to teamcity-feedback[at]jetbrains.com. Also please send teamcity-agent.log from the agent where this slow build executed.

0

Please sign in to leave a comment.