Mercurial unnecessary clone

Hi,
I've found teamcity is disappointingly slow at building my fairly large project from mercurial. It takes 5 minutes to transfer the repository sources

I've investigated and found that teamcity is cloning my repository from its cache in TeamCity\Data\system\caches\mercurial to a temporary repository in TeamCity\temp. It is then doing an update to the right revision, to send to the agent. This clone takes a long time because I have a lot of history in my repository.

Instead, you could use hg archive (which is like svn export) to get the required revision straight from your cached copy of the repository. I've timed, and this takes a quarter of the time for my repository.

7 comments
Comment actions Permalink

Hi Alex,
thanks for the suggestion, I created an issue you can watch/vote for: http://youtrack.jetbrains.net/issue/TW-16796.
Clone of the repository happens when we need to build a full patch, which happens for example if you set an option 'Clean all files before build' to true at the VCS settings of you build configuration or run history builds.
When we build incremental patch we do not clone, we just 'hg cat' some files on specific revision.
If you set option 'Clean all files before build' please ensure that you really need it, without that option building patch should be much faster.

0
Comment actions Permalink

Thanks. I think I'm right in saying that I need to use "clean all files before build" because I don't trust myself not to make a mistake where one of the files of the build isn't created, and I don't want that mistake to go unnoticed because the file already exists from a previous build. Am I right?  I'll consider turning off the setting until the issue is fixed though.  Because the mercurial support is a plugin, does it mean that it can be modified by the community? I'd be very happy to fix it myself.

0
Comment actions Permalink

> Thanks. I think I'm right in saying that I need to use "clean all files  before build" because I don't trust myself not to make a mistake where  one of the files of the build isn't created, and I don't want that  mistake to go unnoticed because the file already exists from a previous  build. Am I right?

This sounds reasonable, but maybe it should be handled by build script: script could clear all results of the previous build. But maybe your case is more complicated thatn just removal of some dir or group of files.

> Because the mercurial support is a plugin, does it mean that it can be  modified by the community? I'd be very happy to fix it myself.

Yep it could be modified, you can get sources at http://hg.jetbrains.org/hg/mercurial/ . It would be great if you attach your fix to the issue.

0
Comment actions Permalink

Hi Alex

Consider using our Swabra feature - it helps to clean files generated during the build.

Michael

0
Comment actions Permalink

I've done the patch to use hg archive. Attached to the issue in youtrack for you to integrate and maintain :)

Even if I use swabra and turn off doing a clean patch every build, our system still does clean patches fairly often, as we have many agents, and the build process modifies some files. So definitely worth using hg archive.

0
Comment actions Permalink

Great! Thank you! I'll take a look at your patch.


0
Comment actions Permalink

An mercurial alternative is to enable the purge extension and then do a

hg purge --all

before your build steps.  That will remove all files not in mercurial.  You might also what to do a hg revert --all to revert any changed files.  The repository should then be exactly as a fresh clone.

Of course this relies on agent side checkout.

0

Please sign in to leave a comment.