Why does a full (slow) git checkout is performed

Hi Lior,

TeamCity only cleans the checkout folder under some circumstances described here: https://confluence.jetbrains.com/display/TCD10/Clean+Checkout

Other than that, we don't do it, and in this case, the build log should be clear about performing a clean checkout. Make sure that you don't have any third party tools that does it, or that you aren't using build features that remove files, such as Swabra (https://confluence.jetbrains.com/pages/viewpage.action?pageId=74844984) or Free Disk Space (https://confluence.jetbrains.com/display/TCD10/Free+disk+space)

In case i run into this again, if i include the full build log, will that help you analyze why it happened?

Hi Lior,

it might help or it might not. It might if it's TeamCity actually removing the data before the build, but you can also check it by yourself, as it will state explicitly that it's cleaning the directory or performing a clean checkout. Usually, this message also includes the reason as to why the clean checkout is being used. If the reference is that the directory is empty, then you can check the previous build that has used that folder and check its last lines for hints as to why it might have been removed. If none of the build logs are mentioning removing anything, then there is some third element acting. It might be another build reusing the same directory, a third party application, or, if you are using cloud profiles with cloud images, the fact that restarting the "machine" will simply set up a new, clean one.

The issue happened again today. here are the logs (from wrapper.log and vcs log files). I changed the repo names BTW.

INFO | jvm 1 | 2018/04/12 08:49:06 | [2018-04-12 08:49:06,391] WARN - jetbrains.buildServer.VCS - Error output produced by: "C:\Program Files\Git\bin\git.exe" submodule init
INFO | jvm 1 | 2018/04/12 08:49:06 | [2018-04-12 08:49:06,391] WARN - jetbrains.buildServer.VCS - Submodule 'Assets/MoonCore' (git@github.com:xxxxxxx/yyyyyyyy.git) registered for path 'Assets/MoonCore'
INFO | jvm 1 | 2018/04/12 08:49:16 | [2018-04-12 08:49:16,959] WARN - jetbrains.buildServer.VCS - Error output produced by: "C:\Program Files\Git\bin\git.exe" -c credential.helper= submodule update --force
INFO | jvm 1 | 2018/04/12 08:49:16 | [2018-04-12 08:49:16,959] WARN - jetbrains.buildServer.VCS - Cloning into 'D:/BuildAgent/work/Sales_android_Unity_553p1/Assets/MoonCore'...

2018-04-12 08:49:03,051] INFO - jetbrains.buildServer.VCS - [D:\BuildAgent\work\Sales_android_Unity_553p1]: "C:\Program Files\Git\bin\git.exe" clean -f -d
[2018-04-12 08:49:03,481] INFO - jetbrains.buildServer.VCS - [D:\BuildAgent\work\Sales_android_Unity_553p1\Assets\MoonCore]: "C:\Program Files\Git\bin\git.exe" clean -f -d
[2018-04-12 08:49:03,597] INFO - jetbrains.buildServer.VCS - [D:\BuildAgent\work\Sales_android_Unity_553p1]: "C:\Program Files\Git\bin\git.exe" submodule init
[2018-04-12 08:49:06,391] WARN - jetbrains.buildServer.VCS - Error output produced by: "C:\Program Files\Git\bin\git.exe" submodule init
[2018-04-12 08:49:06,391] WARN - jetbrains.buildServer.VCS - Submodule 'Assets/MoonCore' (git@github.com:xxxxxxx/yyyyyyyy.git) registered for path 'Assets/MoonCore'
[2018-04-12 08:49:06,391] INFO - jetbrains.buildServer.VCS - [D:\BuildAgent\work\Sales_android_Unity_553p1]: "C:\Program Files\Git\bin\git.exe" submodule sync
[2018-04-12 08:49:07,659] INFO - jetbrains.buildServer.VCS - Update submodules credentials
[2018-04-12 08:49:07,661] INFO - jetbrains.buildServer.VCS - Update credentials for submodule with url git@github.com:xxxxxxx/yyyyyyyy.git
[2018-04-12 08:49:07,661] INFO - jetbrains.buildServer.VCS - Url git@github.com:xxxxxxx/yyyyyyyy.git does not require authentication, skip updating credentials
[2018-04-12 08:49:07,670] INFO - jetbrains.buildServer.VCS - [D:\BuildAgent\work\Sales_android_Unity_553p1]: "C:\Program Files\Git\bin\git.exe" -c credential.helper= submodule update --force
[2018-04-12 08:49:16,959] WARN - jetbrains.buildServer.VCS - Error output produced by: "C:\Program Files\Git\bin\git.exe" -c credential.helper= submodule update --force
[2018-04-12 08:49:16,959] WARN - jetbrains.buildServer.VCS - Cloning into 'D:/BuildAgent/work/Sales_android_Unity_553p1/Assets/MoonCore'...
[2018-04-12 08:49:16,960] INFO - jetbrains.buildServer.VCS - Submodules update in D:\BuildAgent\work\Sales_android_Unity_553p1 is finished in 9298 ms

Hi Lior,

I don't see anything related to a clean checkout beyond submodules failing, but no explicit error reported. What's the error? Is it on the build log?

I keep seeing this issue over and over again. Attaching logs from the last time it happened for one of our builds (see in logs at around 10:34:16.135 in teamcity-agent.log):
https://www.dropbox.com/sh/rklq2ujx5ema6wp/AAAA80uo3WXJy4NuYHRIgQSna?dl=0

It appears that for some reason the VCS checkout process decides that it should run a clean checkout: VCS_SETTINGS_HAVE_BEEN_CHANGED

I have no idea why. this issue is causing our builds to run VERY VERY SLOWLY from time to time. i would really like to understand if this is a bug or what we're doing wrong.

The logging in there seems to imply that you have modified the VCS Settings? Have you modified them? Are the VCS Settings parametrized?

Which settings? And changed since when? I believe there is an audit log I can use to see those changes. But I don't think I've made any changes recently.

Also, what should I be looking for? Changes to the specific vcs root? Or to the current build configuration vcs settings?

I just verified - there were no changes to the VCS roots or to the build configuration. There was a build on this agent, with the same configuration a day before and it did not perform a clean checkout.

What else can i check to diagnose this ?

Denis - can you provide any help? we are still seeing this in the TeamCity build log:

"Will perform clean checkout. Reason: VCS settings have been changed"

The settings were not changed (what is the change? what is considered to be a change? it should be logged).

Also, why does a VCS root change require a clean checkout ?

Also, above in this thread, you asked whether our "VCS settings are parameterized" ? what did you mean by that? maybe that's the problem here?

Hi Lior,

I'm sorry for missing your last comments, I missed the notifications for some reason. A parametrized VCS Root is if you are configuring its options via %xxx% parameters rather than just with the full contents.

May I ask you to send us the build log and teamcity-agent.log (please make sure the agent log covers the timeframe of the build as well). Please use the submit a request button on top of this page for that, attaching the files there will be private and it'll make it easier for us to track the issue itself.

Hi Denis Lapuente

I am working on getting those logs for you, last build we were running was from a few days ago, i don't have the build log, but i do have the agent log (will send it along with the build log i have from TeamCity, not from the agent itself).

Hi Lior,

if you haven't removed/cleaned up the build, you can access the build results, build log tab, click on Download full build log. Otherwise it would be useful if it replicates again, or if you can have any consistent result.

I attached that to my request, but its the build log from TC, not the actual file from the agent's file system (maybe this doesnt matter that much).