Svn agent-side checkout mode is 2-3x slower than svn cli

Running on TeamCity 6.5.3 with an idle svn server and an idle build agent (on seperate machines). Most noticable in my environment on a clean checkout where the svn cli checkout of the repo takes 5mins, and the TeamCity agent-side checkout of the same repo takes 12+minutes.

For long running build agents this is not much of an issue. For cloud build agents with a few build configs with different vcs root settings (that include the big svn vcs root) it means the agent initially spends a lot of time checking out the same large svn repo. This is made worse by the poor performance of the svn agent-side checkout.

I did not find any issues on youtrack for slow agent-side svn checkout. Wondering if anyone else can confirm/deny the difference between svn cli and teamcity agent-side checkout performance in their environment.

5 comments
Comment actions Permalink

Hello Peter,

  Teamcity doesn't use a native svn client for agent side checkout. Instead, it uses SVNKit Java library for this kind of operations.
  This may be the reason of the difference in the checkout speed.

  To check, whether TeamCity adds overhead for the checkout operations, you can download SVNKit library from http://svnkit.com/download.php, unpack it, and run checkout command using command-line utility, included into SVNKit distribution. But, there shouldn't be a noticeable difference. If there is, please share your findings with us.

  Kind regards,
  KIR

0
Comment actions Permalink

Here are the results of a quick & dirty benchmark run:


++ which jsvn
/tmp/svnkit-1.3.5.7406/jsvn




++ rm -rf repo
++ jsvn co --quiet https://svn/project/trunk repo
real    10m21.311s
user    5m10.760s
sys     2m10.630s
++ du -hs repo
2.5G    repo


++ rm -rf repo
++ svn co --quiet https://svn/project/trunk repo
real    7m14.852s
user    1m44.550s
sys     1m5.250s
++ du -hs repo
2.5G    repo


...
[12:31:35]: Updating sources: agent side checkout... (15m:02s)
[12:31:35]: [Updating sources: agent side checkout...] Will perform clean checkout. Reason: Agent doesn't have any version of the project sources
[12:31:35]: [Updating sources: agent side checkout...] Cleaning <checkout_dir>
[12:32:11]: [Updating sources: agent side checkout...] VCS Root: svn (14m:20s)
[12:32:11]: [VCS Root: svn] checkout rules: project/trunk=>repo; revision: 16711_2011/09/27 16:12:33 +0000
[12:32:11]: [VCS Root: svn] Will use fast SVN update
[12:46:32]: [Updating sources: agent side checkout...] VCS Root: git
...


## Server-side checkout of same repo when there is a cached clean patch
[13:14:34]: Updating sources: server side checkout... (2m:52s)
[13:14:34]: [Updating sources: server side checkout...] Will perform clean checkout. Reason: Agent doesn't have any version of the project sources
[13:14:34]: [Updating sources: server side checkout...] Transferring cached clean patch for VCS root: svn
[13:16:36]: [Updating sources: server side checkout...] Transferring repository sources: 691.16Mb so far...
[13:17:11]: [Updating sources: server side checkout...] Repository sources transferred: 917.40Mb total
[13:17:11]: [Updating sources: server side checkout...] Average transfer speed: 5.86Mb per second
[13:17:11]: [Updating sources: server side checkout...] Removing <checkout_dir>/repo
[13:17:16]: [Updating sources: server side checkout...] Updating <checkout_dir>/repo



Unfortunately cannot use server-side checkout as projects in the repo make heavy use of links (http://youtrack.jetbrains.net/issue/TW-5953).
0
Comment actions Permalink

Hello Peter,

  I'm sorry for delay with the answer - I was on vacation the last week.

  Thanks a lot for the benchmarks - they are really useful. How many externals are there in your VCS root?

   I've filed an issue with your report: http://youtrack.jetbrains.net/issue/TW-18514 - please answer there.

  Thanks!
  KIR

0
Comment actions Permalink

This issue is killing me right now.  I'm seeing a similar performance difference.

I checked the ticket, but there hasn't been any movement on it AFAICT.  

Is it planned in an upcoming sprint?  Where is it in the queue?

0
Comment actions Permalink

Hello,

  Please provide details of your setup and your benchmarks as a comment to the issue http://youtrack.jetbrains.net/issue/TW-18514
  I hope I'll have time to fix it in TeamCity 7.0.

  Kind regards,
  KIR

0

Please sign in to leave a comment.