Error while applying patch using git

I am using TeamCity 6.5.6

Recently started using agent side checkouts instead of server side checkout.

I am getting this message often now:

hecking for changes
[14:19:44]: Clearing temporary directory: C:\TeamCity\buildAgent\temp\buildTmp
[14:19:44]: Checkout directory: D:\Code\NoRanorex
[14:19:44]: Updating sources: server side checkout... (14s)
[14:19:45]: [Updating sources: server side checkout...] Will perform clean checkout. Reason: Agent doesn't have any version of the project sources
[14:19:45]: [Updating sources: server side checkout...] Building clean patch for VCS root: git-noranorex
[14:19:53]: [Updating sources: server side checkout...] Repository sources transferred: 84.66Mb total
[14:19:54]: [Updating sources: server side checkout...] Removing D:\Code\NoRanorex
[14:19:59]: [Updating sources: server side checkout...] Updating D:\Code\NoRanorex
[14:19:59]: [Updating sources: server side checkout...] Error while applying patch: Failed to create directory: D:\Code\NoRanorex
[14:19:59]: Publishing internal artifacts
[14:19:59]: [Publishing internal artifacts] Sending build.finish.properties.gz file
[14:19:59]: Build failed to start. Artifacts will not be published for this build
[14:19:59]: Build finished

My configuration is 1 TeamCity server (which is also the agent).

I have 1 build currently with 2 configurations.
Both configurations share the same VCS root and checkout to the same folder.

What can be the cause of this issue ? is this a bug?

Any help will be appreciated!

Thanks
Lior
13 comments
Comment actions Permalink

Hi Lior,

what are the settings of you git VCS root? Does that error occur in every build? BTW, this log says you still use server-side checkout.

0
Comment actions Permalink

You are right.

I had one configuration with agent side and one with server-side. They are both agent side now, getting the same error:

[16:47:33]: Checking for changes
[16:47:34]: Clearing temporary directory: C:\TeamCity\buildAgent\temp\buildTmp
[16:47:34]: Checkout directory: D:\Code\NoRanorex
[16:47:34]: Updating sources: agent side checkout... (3s)
[16:47:34]: [Updating sources: agent side checkout...] Will perform clean checkout. Reason: Agent doesn't have any version of the project sources
[16:47:34]: [Updating sources: agent side checkout...] Cleaning D:\Code\NoRanorex
[16:47:37]: [Updating sources: agent side checkout...] Failed to perform checkout on agent: Failed to create build checkout directory: D:\Code\NoRanorex
[16:47:37]: Publishing internal artifacts
[16:47:37]: [Publishing internal artifacts] Sending build.finish.properties.gz file
[16:47:37]: Build failed to start. Artifacts will not be published for this build
[16:47:37]: Build finished

This happens only once. if i maunally start the build again it works in most cases.

Am i doing something wrong ? what could be the cause?
0
Comment actions Permalink

It seems like directory D:\Code\NoRanorex is blocked by some other process. Do you have an antivirus installed on this machine?

0
Comment actions Permalink

There is an antivirus but i don't think it is the issue.

Attempting to delete the folder manually works.

0
Comment actions Permalink

Also, starting the build again works fine the 2nd time. If there was a process still holding on the folder/file, it would probably fail the 2nd time as well.

0
Comment actions Permalink

I would suggest to take a look at conflicting software topic in out documentation: http://confluence.jetbrains.net/display/TCD65/Known+Issues#KnownIssues-ConflictingSoftware

0
Comment actions Permalink

I don't think this is caused by a conflicting software since pretty much nothing else except for the server + agent is running.

Could it be that the agent itself holds some resources from previous run and fails the next build?


I mentioned that i am using the same checkout dir with all our builds (2 build configurations). Not sure if this is a safe practice or not, but it may be the cause of the issue?

0
Comment actions Permalink

There are no known issues about stale processes left by agent in the checkout directory. The fact that you can remove this directory manually proves that the problem is not caused by such processes.
Also if they were, you could see them with help of sysinternals process explorer.

From the other hand antivirus software is the most common thing to blame in such case. For example, the common workaround used by many programs is to try to remove file several times...
Unfortunately this workaround does not work in general case, becase  there are many libraries which do not remove files in such a way, and  they will fail if antivirus blocks access to a directory even for a  short time.

More on problems with antivirus software:
http://subversion.apache.org/faq.html#windows-access-denied
http://xiaonanji.wordpress.com/2008/10/06/svn-access-denied-problem/
http://stackoverflow.com/questions/119816/antivirus-and-file-access-conflict-good-programming-practices

0
Comment actions Permalink

Just updating that none of the suggestions helped out.

Still experiencing this issue on a daily basis (every other build fails to start).

Note that 2 main changes were introduced recently that may have caused it:

  1. Upgrade to TeamCity 6.5.6
  2. Change from Server side checkout --> Agent side checkout.


Except for these, the build configuration remained IDENTICAL.

We have performed over 400 builds prior to these 2 changes, all of them worked and never this issue occured.

What else can be checked? i am not sure when it will be possible for us to do a downgrade to test this.

Thanks
Lior

0
Comment actions Permalink

Hello,

Please check build agent windows service is running under proper account and you have a non-network mapped drive D on the build agent

0
Comment actions Permalink

Hi Lior

Is the issue reproduced now?
Could you please run Sysinternals ProcMon tool, and monitor processes that open handles in this directory.

0
Comment actions Permalink

This issue does not occur anymore, however it may be a bug after all.

I had a weird combination of 2 VCS roots (by mistake) that had a pretty similar name.

Both used the same folder for checking out the repository (both used the same Git remote repository + branch).

Do you think this is a bug ?

Thanks
Lior

0
Comment actions Permalink

This is a normal scenario for TeamCity to have more than one VCS root for one configuration. We introduced checkout rules to make it easy to setup.
The right use-case it to avoid mixing of 2 version controls.

Please create an issue in our issue tracker for that at http://youtrack.jetbrains.net

0

Please sign in to leave a comment.