Updating sources is slow

Behold the following log snippet:



[12:17:38]Updating sources: server side checkout (1h:17m:45s)
[12:17:39][Updating sources] Will perform clean checkout. Reason: The project sources on the agent are newer than requested
[12:17:39][Updating sources] Building clean patch for VCS root: **********
[12:19:40][Updating sources] Transferring repository sources: 796.19 MB so far...
[12:21:40][Updating sources] Transferring repository sources: 1.15 GB so far...
[12:23:40][Updating sources] Transferring repository sources: 1.39 GB so far...
[12:24:23][Updating sources] Building clean patch for VCS root: ************
[12:25:43][Updating sources] Transferring repository sources: 1.63 GB so far...
[12:27:46][Updating sources] Transferring repository sources: 1.79 GB so far...
[12:29:50][Updating sources] Transferring repository sources: 2.10 GB so far...
[12:31:52][Updating sources] Transferring repository sources: 2.25 GB so far...
[12:34:06][Updating sources] Transferring repository sources: 2.35 GB so far...
[12:36:15][Updating sources] Transferring repository sources: 2.46 GB so far...
[12:37:28][Updating sources] Transferring cached clean patch for VCS root: **************
[12:38:16][Updating sources] Transferring repository sources: 3.10 GB so far...
[12:39:55][Updating sources] Building incremental patch over the cached patch
[12:39:55][Updating sources] Repository sources transferred: 3.58 GB total
[12:39:55][Updating sources] Average transfer speed: 2.74 MB per second
[12:40:19][Updating sources] Removing C:\BuildAgent\work\fe604fd3b9fcd4ad
[12:48:51][Updating sources] Updating C:\BuildAgent\work\fe604fd3b9fcd4ad
[13:35:24]Swabra (50s)
[13:35:24][Swabra] Saving C:\BuildAgent\work\fe604fd3b9fcd4ad directory state to snapshot file fbf2297d.snapshot





The issue isn't the time it takes to get 3.5 GB to the builder (although that's pretty darn slow, too), as that's been re-hashed on the forums before.  The issue we have is the amount of time it takes to do the final step.  'Updating' takes 50mins or longer.  This is typical for us, and we don't see a way around it besides using client side checkouts, which we are actively trying to avoid.




Can we get some better logging here to tell what the issue is?  Are others seeing this same behavior?  Is this unique to perforce, or are others seeing this, too?
4 comments
Comment actions Permalink

2 years passed and still the issue persists, plus logging didn't change. This below from 9.1.6:

[19:01:17]Checkout directory: C:\TeamCity\buildAgent\work\e500fbe33724f0d0
[19:03:17]Transferring repository sources: 155.5 MB so far...
[19:05:17]Transferring repository sources: 419.12 MB so far...
[19:07:17]Transferring repository sources: 573.06 MB so far...
[19:01:17]Updating sources: server side checkout (15m:22s)
[19:08:32][Updating sources] Building incremental patch over the cached patch
[19:09:24][Updating sources] Transferring repository sources: 818.31 MB so far...
[19:09:26][Updating sources] Repository sources transferred: 818.9 MB total
[19:09:26][Updating sources] Average transfer speed: 1.67 MB per second
[19:09:26][Updating sources] Removing C:\TeamCity\buildAgent\work\e500fbe33724f0d0
[19:09:26][Updating sources] Updating C:\TeamCity\buildAgent\work\e500fbe33724f0d0
[19:16:39]Step 1/5: NuGet Installer
[19:16:39]Step 2/5: Clean temp build dir (Command Line)
0
Comment actions Permalink

We're having the same issue:

[13:16:16]Transferring repository sources: 28.31 MB so far...
[13:18:16]Transferring repository sources: 188.43 MB so far...
[13:20:16]Transferring repository sources: 468.87 MB so far...
 
But only on linux agents (Ubuntu Server 14.04) Server is running on Mac OS X. Network connection between agent and server is fine (checked with iperf)
 
0
Comment actions Permalink

Yes, we're having the same issues. Has anyone managed to work out a solution for this as we're seeing the same problem ever day and when we hit this issue it adds 50 minutes to a build which normally takes 10 minutes.

0
Comment actions Permalink

Please provide a solution for this , We are facing the same issue . 

Updating sources: server side checkout (8m:06s)
[01:23:01][Updating sources] Building non-cacheable clean patch for VCS root: //ProjectUI/F138896__First_Version; checkout rules: whitelisting=>; revision: 2709414
[01:25:01][Updating sources] Transferring repository sources: 24.43 MB so far...
[01:27:01][Updating sources] Transferring repository sources: 48.5 MB so far...
[01:29:02][Updating sources] Transferring repository sources: 73.12 MB so far...
[01:31:02][Updating sources] Repository sources transferred: 98.23 MB total
[01:31:02][Updating sources] Average transfer speed: 209.48 KB per second
[01:31:02][Updating sources] Removing /teamcity_workspace/BA/F138896__First_Version
[01:31:02][Updating sources] Updating /teamcity_workspace/BA/F138896__First_Version

0

Please sign in to leave a comment.