"Getting Project Sources" very slow

Hi,

I am running TC 3.1.2 (Build 6881) on a Win2K8 x64 server, which connects to a remote Subversion repository over svn+ssh.

This has started some times ago on one of my projects but I now have the issue on all projects.
The "Getting Project Sources" can take 30 to 60 mins, while it only takes 5 mins to get all the project from a SVN client.

I have seen few posts with people having similar problems. What could cause this issue? Could you give me some guidance on where to look for the problem?

Thank you
Julien

7 comments
Comment actions Permalink

Hello Julien,

Do you reuse this VCS root between several build configurations?
Do you use checkout rules?

If yes, please try creating a build configuraiton (may be a fake one) which would collect changes for the VCS root as a whole.

Kind regards,
KIR

0
Comment actions Permalink

I encountered same problem - it seems to be caused by usage of check-out rules - at least without it checkout works much more quick (1 minute for full checkout vs. 10 minutes for update).

I can't follow your advice, because I have a build scripts folder placed in the root of repository - and it seems for a not good idea to check out all tags and branches as well for a build of trunk.

0
Comment actions Permalink

Hi Kirill,

Thank you for your prompt response and sorry for not getting back to you earlier but I was on holidays since ;)

I have created a new VCS root as you suggested as it was shared by 3 projects. However it did not change anything, it still takes almost one hour to get the project sources.
We are not using checkout rules.

I have another project using a different VCS root not shared, for which I have the same problem. It was taking a couple of seconds and now it takes 7 or 8 minutes to get the sources.
Looking at the history of this project, it seems that the problem started with the upgrade to 3.1.2

I have attached 2 screen shots with the VCS root and project settings.

I would really appreciate if you could help.

Best
Julien



Attachment(s):
Project Version Control Settings.jpg
VCS Root Settings.jpg
0
Comment actions Permalink

Hello Julien

Could you please collect all server debug logs for your setup:
- stop TeamCity server
- remove TeamCity/log/*
- enable debug logging as described at http://www.jetbrains.net/confluence/display/TCD3/Reporting+Issues#ReportingIssues-logging
and please enable logging for teamcity-svn.log (see below on the same page).
- start the server, and run builds with slow checking for changes phase
- zip TeamCity/log directory and send it to kir jetbrains.com

I'd also really appreciate if you test your configuration with TeamCity 4.0 build, because, in fact, there was a lot of changes in this area and
we'd like to get real-world feedback on it.

Thanks!
KIR

0
Comment actions Permalink

Hi Kir

Sorry for not replying earlier but Monday/Tuesday were closed in France.
I have enabled logging and have sent you the logs by email as requested. I hope I got it properly setup....

I have seen the TC 4.0 beta but I did not really want to install it on my build server yet.
Is it really stable enough to use it in an environment other than a test environment?

Else I might be able to get the separate server to run it but it will take a bit of time...

Thanks
Julien

0
Comment actions Permalink

Hello Julien,

I've got your logs, thanks. It really looks that checking for changes phase is rather slow in your setup.

I can suggest a couple of things:
- increase default VCS poll timeout for the server (for now it is 60 seconds, I think it should be increased in your case, because each check for changes takes > 20 seconds for each VCS root).
- try to profile project building with TeamCity using a plugin from http://www.jetbrains.net/confluence/display/TW/Server+Profiling .

Regarding TeamCity 4.0. Internally in JetBrains, we're constantly running this new version, moreover, it is supposed to be released in this year.
I'd recommend to install a one test installation of TC4.0, create one SVN project for it and check how it works. If you wont encounter problems,
you can use it in production. If you find problems, tell us.

To be honest, it is much more likely that we will fix problems in 4.0 (and give a patch, if possible), than provide a patch for 3.1.x branch.
In the meanwhile, I'll try to repeat your setup locally.

Kind regards,
KIR

0
Comment actions Permalink

Hi Kir,

I will try your suggestions and see if it helps speeding the process. I had checked the OS performance counters of the server during the build and it did not seem overloaded.

In the meantime, I have tried again your previous suggestion of using a new different VCS root. The first run took even longer than before (1h50), but the next builds ran in a range of 15 to 30 minutes. I then also tried disabling the "Clean all files before build" option. The next couple of builds ran in less than 5 minutes (we enabled the option as some files were not always properly updated, hence causing the builds to fail)

The IT team is setting up a new server, so I will be able to setup a TC4.0 server and test how it goes.
Though we are close to a release date so I am not sure how much time I will be able to spend on this but I hope I can take care of this next week...

I will keep you updated.

Best regards
Julien

0

Please sign in to leave a comment.