Issues with TeamCity - TFS

Hi,

We faced a really serious/strange issue with Teamcity today. One of our build agents ran out of disk space and it started throwing this exception

System.Exception: The cache file C:\Users\tfsbuild\AppData\Local\Microsoft\Team Foundation\4.0\Cache\VersionControl.config is not valid and cannot be loaded.

Please correct or delete the file.

---> Microsoft.TeamFoundation.VersionControl.Client.VersionControlException: The cache file C:\Users\tfsbuild\AppData\Local\Microsoft\Team Foundation\4.0\Cache\VersionControl.config is not valid and cannot be loaded.

We freed up disk space on the build agent but the problem, won’t go away, so we deleted the cache on the build agent and restarted both the build agent and teamcity server. After we restarted TeamCity server, it on its own scheduled around 1300 builds. That was not that big of deal as we removed everything from queue.

The serious issue was that any builds that we tried to run later were not checking out the latest code from TFS. The builds were tied to revisions in TFS that were checked in months ago, some even from last year i.e. the latest revision in TFS according to TeamCity was incorrect.  We found this out by luck as we have recently changed our build scripts and we saw that the teamcity was passing in new-variables to scripts but the scripts failed to accept the variables. Had we not changed the build scripts, we might have shipped months old code.

However, we found that checking in something to TFS, made teamcity get the correct latest revision from TFS. However we had to do dummy checkins for reach VCS root.

Can you elaborate what could have gone wrong on our machines, so that we know next what to do if this happens next time.

1 comment
Comment actions Permalink

Hi, after queueing a build TC polls all VCS roots that participate in the build to get the latest revision # (the revision the build will be run against). So, it can either be TC with inconsistent information or TFS that returns old data.

You mentioned that after restart TC scheduled 1300 builds. Are you using per-check-in trigger?

Also, are you using server-side or agent-side checkout?

Please also attach teamcity-server.log and teamcity-agent.log

0

Please sign in to leave a comment.