TeamCity - Subversion - Working Directory / Sources Caches

Hello

I have a query regarding the mechanism that VCS Source work, specifically Subversion sources. Just to put things into context...

-          We have TeamCity Enterprise 7.1.5 (build 24400) environment. We have a quite a large number of mainly .Net based Builds.

-          All VCS Roots, which is SVN  based, are configured to be checked out Automatically on server.

-          We have set the Clean all files before build: OFF

-          We make use of the Swabra plugin to ensure that the Working Directories stay clean, without having to explicit refresh the SVN checkout. Config is as follows:

     -          File cleanup : After build finish

     -          clean checkout : disable

     -          I had queried about the Swabra mechanism previously...

                http://devnet.jetbrains.com/thread/447738

We make extensive use of templates. There is one VCS root, with the SVN branch set by a paramater. The Build Configurations are templated and for each new branch, we simply create a new TeamCity Project from a Template Project.

Regarding the Working Directories, I am aware that this dependent on the VCS root and Checkout Rules. I can see that where the VCS Root and Checkout Rules are the same, various Build Configuration in TeamCity, will use the same Checkout Directory on the Agent.

However, I am not quite sure about SVN caches work on the Server side, the ones located at
*\TEAMCITY\.BuildServer\system\caches\sources
I have been monitoring the cache files that are generated within that location and as far as I can tell, it is creating a new one for each individual build config, even those that have the same VCS Root & Checkout rules. I have looked within the version.map file and it appears that the only difference between the chache file is the timestamp, but otherwise the same revision number and file size are the same.
So I want to undertand what the mechanism is that is used to generate these Source Chache files. The main reason it is an issue for us, that over time we end up generating a large number of these cache files and use up all the available storage on the server. And from what I've seen, there is no mechanism to force a cleanup of these caches on the server side.

Any help or clarification on this would be appreciated.

Many Thanks,

RG

5 comments
Comment actions Permalink

Hello RG,

   These server side source caches should be cleaned during the regular build server cleanup process. Have you configured it?

   From the UI, you can enforce cleaning these caches for a particular build configuration using "Enforce clean checkout" action from the build configuration "Actions" menu in toolbar.

   Kind regards,
   KIR

0
Comment actions Permalink

Hello Kir,

Thanks for your response.

We do have a nightly cleanup process setup and so the cache files do get cleared up on a nightly basis. However there are busy periods, when many branches are under active development and it results in space running out before the the cleanup process is called. This is because so many Source Caches get generated that they use up all the available space on the TeamCity server.

That is why i was inquiring about the scope of each individual Cache. It appears to me that even when the VCS root and checkout rules are the same across multiple Build Configuration, it still creates a separate cache file for each build. Is this by design or could I have misconfigured the TeamCity Environment in some way.

Thanks,

RG

0
Comment actions Permalink

Hello,

   Actually, if you have exactly the same specification of VCS Root and same checkout rules - the source caches should be reused.

   Please pay attention, that if you're using variables in VCS Root specification, and these variables have different values for different configurations - different cache files will be created. This is by design.

   Another thing you may try in your situation - using agent-side checkout. This may be efficient, if you don't use exclude checkout rules a lot.

   Hope this helps,
   KIR

0
Comment actions Permalink

Hello KIR,

Thanks for your response.

In our case we have common VCS root, with only the branch part parameterised. Then each project (with multiple Build Configs) is providing the same branch name, hence all Build Configs within a project should have the same VCS root. There are no Checkout rules. However, we do make use of VCS Rules on the Build Trigger.
Is it possible that the Build Trigger Rules are causing the multiple Source Cache files, for what appears to be the same VCS Root.

Thanks,

RG

0
Comment actions Permalink

Hello,

   I'm sorry for the delay in the communications.

   I've just tried to reproduce the problem - i.e. I've created a VCS root with a parameters, and created 2 build configurations with the same value of the parameter. 2 builds for these 2 configurations did reuse the VCS Root cache.

  From the code, I see a possibility when the cache won't be reused and copies would be created - when you create a copy of the VCS Root with exactly the same parameters as a previous one. The matter is, VCS Root ID is included into the cache key, so you may have 2 identical caches for 2 VCS Roots.

  But as far as I understand, whole project reuses the same VCS Root in your case, and there are no duplicates, correct?

  To investigate further, we may need debug VCS logs for a situation, when 2 consequent builds with same VCS settings are run and cache is not reused between them.

  Kind regards,
  KIR

0

Please sign in to leave a comment.