Builds always cleanup after themselves

We started to experience a strange behavior on the Teamcity Server the last 2 weeks.

All our build definitions now start with a
"Reason: Checkout directory is empty or doesn't exist"

So they dont do a "get latest"/diff anymore but clean/full checkout

So our build times went up from 1m30 or so to 9 minute.
(Those are the builds that do a deploy with robocopy and since the timestamp always changes all the files are copied over)

I notice that the working folder of the build agent like below get cleanud up *imemdiatly* after a build
\\atoma\h$\TeamCity\buildAgent\work\b485cefad20659b9\

But we don't have a task/buildstep that says to cleanup after itself.
Also the server is not in need of diskspace :
C: 16,7 free of 59,8
H: 12,2 free of 29,9

Since we didn't make a change to TC (we can see this from the audit) log it still seems that some threshold is reached without human doing.

We looked through the server logs but could not find a clue:

\\atoma\TeamCity\logs\teamcity-server.log
\\atoma\TeamCity\logs\teamcity-activities.log
\\atoma\TeamCity\logs\teamcity-vcs.log
\\atoma\TeamCity\logs\teamcity-xmlrpc.log
\\atoma\TeamCity\logs\teamcity-cleanup.log
\\atoma\TeamCity\logs\teamcity-tfs.log
\\atoma\TeamCity\logs\teamcity-tfs.log.1

This kept us busy for 0.5 days so are interested in what the reason can be.

Thanks for your help

Tom

7 comments
Comment actions Permalink

Please check conf/buildAgent.properties for existence of teamcity.agent.build.checkoutDir.expireHours property. Also, please post work/directory.map file content.

0
Comment actions Permalink

\\atoma\TeamCity\buildAgent\conf\buildAgent.properties 395 b 15/07/2013 15:12 -a--


#Generated properties (see buildAgent.dist.properties for description)
#Tue Feb 26 16:05:50 CET 2013
name=Atoma
ownPort=9090
serverUrl=http\://localhost\:8081
workDir=H\:\\TeamCity\\buildAgent\\work
tempDir=H\:\\TeamCity\\buildAgent\\temp
#User defined properties
#Tue Feb 26 16:05:50 CET 2013
env.TEAMCITY_JRE=C\:\\TeamCity\\jre

authorizationToken=093c4e69xxxxxxxxxxxxxxxxxxxxx

\\atoma\h$\TeamCity\buildAgent\work\directory.map 793 b 13/05/2014 13:43 -a--


#Don't edit this file!
#May 13, 2014 1:43:35 PM CEST
--version=2
bt20=CliC::Release V4.2 -> 59d10c1e2d2193ba |?| May 5, 2014 3:17:56 PM CEST |:| default
bt11=CliC::Main Nightly (Clean Checkout with no Deploy) -> dd6a2430864d0873 |?| May 13, 2014 2:00:03 AM CEST |:| default
bt8=Other::Swift -> 8211e862d78bc87f |?| May 13, 2014 2:09:11 AM CEST |:| default
bt7=Other::Toolkit.Essential NuGet Package -> 8d46ce040ed3abe2 |?| May 13, 2014 2:09:46 AM CEST |:| default
bt5=CliC::Development - Morning,Noon,Evening (Clean Checkout, Sandcastle, FxCop,  ...) -> 39dc7bc29a738d28 |?| May 13, 2014 12:15:19 PM CEST |:| default
bt13=Intranet::Intranet -> 5e17e01b6b692331 |?| May 13, 2014 1:34:12 PM CEST |:| default
bt2=CliC::Development -> 39dc7bc29a738d28 |?| May 13, 2014 1:41:00 PM CEST |:| default



I was aware of the teamcity.agent.build.checkoutDir.expireHour property but its not there right.
Notice the timestamps on the file. The problem started on 22/5 and the stamps are before that.

Let me know if I can check a setting or lookup a file. Thanks for the help so far

Regards, Tom

0
Comment actions Permalink

build agent considers all checkout directories in working directory older than 2 weeks as expired and removes them.

The question is, though, if you ran new builds on this agent why didn't checkout dirs appear in directory.map file.

0
Comment actions Permalink

It was a security issue, directoty map was not writeable to the agent and the agent changed accounts 3 weeks ago or so

Would I have been able to disocver this in one of the logs?

Thanks for the help

0
Comment actions Permalink

I searched the logs in the teamcity folder for directory.map but could not find an occurance
I could not find a specific log file in the agent folder

0
Comment actions Permalink

The primary log file for agent is teamcity-agent.log.

0
Comment actions Permalink

yep, it was logged as a warn:

[2014-06-01 15:45:14,543]   WARN - es.DirectoryMapPersistanceImpl - Failed to save directory.map to H:\TeamCity\buildAgent\work\directory.map. H:\TeamCity\buildAgent\work\directory.map (Access is denied)

there was also mention of the error in
\\atoma\TeamCity\buildAgent\logs\wrapper.log

0

Please sign in to leave a comment.