Error collecting changes for VCS root

RE: TeamCity Enterprise 6.5.1 (build 17834)

All,

  Last week, we ran into a problem with TeamCity not being able to do builds, because of the error pasted below my message.  The problem magically disappeared, but now it's back again.  We cannot figure out what is going on.  We're not aware of any changes that we've made to cause this problem, and we've tested the following, all of which work:

  1. Using the Test Connection in the VCS root
  2. Opening a web browser on the TeamCity server (using a VNC connection) and opening a connect to https://oursvnserver/svn/trunk and logging in as the user that Teamcity is using for the builds.
  3. Using curl from the command line to open a connection to https://oursvnserver/svn/trunk


Basically, the only time/place things seem to be failing are from TeamCity.  We've tried restarting TeamCity to no avail.  We're able to check out and check into SVN from our local hosts just fine.  Basically, we cannot seem to isolate what is going wrong here.  I'm thoroughly baffled by how/why the Test Connection in the VCS roots is working but we're still getting the error below.  If anyone has any suggestions, they would be much appreciated.

jetbrains.buildServer.vcs.VcsException: Error collecting changes for VCS root 'Trunk - SVN'
org.tmatesoft.svn.core.SVNAuthenticationException: svn: Authentication required for '<https://oursvnserver:443> MyCompany'
at jetbrains.buildServer.buildTriggers.vcs.svn.SvnSupport$SvnIncludeRuleChangeCollector.collectChanges(SvnSupport.java:551)
at jetbrains.buildServer.buildTriggers.vcs.VcsChangesLoader$1.collectBuildChanges(VcsChangesLoader.java:1)
at jetbrains.buildServer.vcs.VcsSupportUtil.collectBuildChanges(VcsSupportUtil.java:54)
at jetbrains.buildServer.buildTriggers.vcs.VcsChangesLoader.collectChangesByIncludeRules(VcsChangesLoader.java:22)
at jetbrains.buildServer.buildTriggers.vcs.VcsChangesLoader.collectChanges(VcsChangesLoader.java:57)
at jetbrains.buildServer.buildTriggers.vcs.VcsChangesLoader.collectBuildChanges(VcsChangesLoader.java:78)
at jetbrains.buildServer.buildTriggers.vcs.VcsChangesLoader.collectChangesForCheckoutRules(VcsChangesLoader.java:119)
at jetbrains.buildServer.buildTriggers.vcs.VcsChangesLoader.collectChanges(VcsChangesLoader.java:31)
at jetbrains.buildServer.buildTriggers.vcs.VcsChangesLoader._loadChangesForRoot(VcsChangesLoader.java:81)
at jetbrains.buildServer.buildTriggers.vcs.VcsChangesLoader.loadChangesForRoot(VcsChangesLoader.java:169)
at jetbrains.buildServer.buildTriggers.vcs.VcsChangesLoader.loadChangesForRoot(VcsChangesLoader.java:156)
at jetbrains.buildServer.vcs.impl.VcsManagerImpl.loadChanges(VcsManagerImpl.java:76)
at jetbrains.buildServer.serverSide.impl.auth.SecuredVcsManager.loadChanges(SecuredVcsManager.java:11)
at jetbrains.buildServer.serverSide.impl.VcsModificationChecker$1.run(VcsModificationChecker.java:17)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: org.tmatesoft.svn.core.SVNAuthenticationException: svn: Authentication required for '<https://oursvnserver:443> MyCompany'
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.authenticationFailed(SVNErrorManager.java:47)
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.authenticationFailed(SVNErrorManager.java:41)
at org.tmatesoft.svn.core.internal.wc.DefaultSVNAuthenticationManager.getNextAuthentication(DefaultSVNAuthenticationManager.java:214)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:565)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:280)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:268)
at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.doPropfind(DAVConnection.java:126)
at org.tmatesoft.svn.core.internal.io.dav.DAVUtil.getProperties(DAVUtil.java:73)
at org.tmatesoft.svn.core.internal.io.dav.DAVUtil.getBaselineInfo(DAVUtil.java:211)
at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.checkPath(DAVRepository.java:222)
at jetbrains.buildServer.buildTriggers.vcs.svn.RepositoryChangesProvider.askServerForNodeKind(RepositoryChangesProvider.java:149)
at jetbrains.buildServer.buildTriggers.vcs.svn.RepositoryChangesProvider.getNodeKind(RepositoryChangesProvider.java:139)
at jetbrains.buildServer.buildTriggers.vcs.svn.SvnConnection$AddChangesProcessor.getNodeKind(SvnConnection.java:1177)
at jetbrains.buildServer.buildTriggers.vcs.svn.SvnConnection$AddChangesProcessor.getFileChanges(SvnConnection.java:1073)
at jetbrains.buildServer.buildTriggers.vcs.svn.SvnConnection$AddChangesProcessor.collectChangedFileDetails(SvnConnection.java:1020)
at jetbrains.buildServer.buildTriggers.vcs.svn.SvnConnection$AddChangesProcessor.collect(SvnConnection.java:1000)
at jetbrains.buildServer.buildTriggers.vcs.svn.SvnConnection.addChanges(SvnConnection.java:205)
at jetbrains.buildServer.buildTriggers.vcs.svn.SvnSupport$SvnIncludeRuleChangeCollector.collectChanges(SvnSupport.java:526)
... 19 more
org.tmatesoft.svn.core.SVNAuthenticationException: svn: Authentication required for '<https://oursvnserver:443> MyCompany'
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.authenticationFailed(SVNErrorManager.java:47)
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.authenticationFailed(SVNErrorManager.java:41)
at org.tmatesoft.svn.core.internal.wc.DefaultSVNAuthenticationManager.getNextAuthentication(DefaultSVNAuthenticationManager.java:214)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:565)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:280)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:268)
at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.doPropfind(DAVConnection.java:126)
at org.tmatesoft.svn.core.internal.io.dav.DAVUtil.getProperties(DAVUtil.java:73)
at org.tmatesoft.svn.core.internal.io.dav.DAVUtil.getBaselineInfo(DAVUtil.java:211)
at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.checkPath(DAVRepository.java:222)
at jetbrains.buildServer.buildTriggers.vcs.svn.RepositoryChangesProvider.askServerForNodeKind(RepositoryChangesProvider.java:149)
at jetbrains.buildServer.buildTriggers.vcs.svn.RepositoryChangesProvider.getNodeKind(RepositoryChangesProvider.java:139)
at jetbrains.buildServer.buildTriggers.vcs.svn.SvnConnection$AddChangesProcessor.getNodeKind(SvnConnection.java:1177)
at jetbrains.buildServer.buildTriggers.vcs.svn.SvnConnection$AddChangesProcessor.getFileChanges(SvnConnection.java:1073)
at jetbrains.buildServer.buildTriggers.vcs.svn.SvnConnection$AddChangesProcessor.collectChangedFileDetails(SvnConnection.java:1020)
at jetbrains.buildServer.buildTriggers.vcs.svn.SvnConnection$AddChangesProcessor.collect(SvnConnection.java:1000)
at jetbrains.buildServer.buildTriggers.vcs.svn.SvnConnection.addChanges(SvnConnection.java:205)
at jetbrains.buildServer.buildTriggers.vcs.svn.SvnSupport$SvnIncludeRuleChangeCollector.collectChanges(SvnSupport.java:526)
at jetbrains.buildServer.buildTriggers.vcs.VcsChangesLoader$1.collectBuildChanges(VcsChangesLoader.java:1)
at jetbrains.buildServer.vcs.VcsSupportUtil.collectBuildChanges(VcsSupportUtil.java:54)
at jetbrains.buildServer.buildTriggers.vcs.VcsChangesLoader.collectChangesByIncludeRules(VcsChangesLoader.java:22)
at jetbrains.buildServer.buildTriggers.vcs.VcsChangesLoader.collectChanges(VcsChangesLoader.java:57)
at jetbrains.buildServer.buildTriggers.vcs.VcsChangesLoader.collectBuildChanges(VcsChangesLoader.java:78)
at jetbrains.buildServer.buildTriggers.vcs.VcsChangesLoader.collectChangesForCheckoutRules(VcsChangesLoader.java:119)
at jetbrains.buildServer.buildTriggers.vcs.VcsChangesLoader.collectChanges(VcsChangesLoader.java:31)
at jetbrains.buildServer.buildTriggers.vcs.VcsChangesLoader._loadChangesForRoot(VcsChangesLoader.java:81)
at jetbrains.buildServer.buildTriggers.vcs.VcsChangesLoader.loadChangesForRoot(VcsChangesLoader.java:169)
at jetbrains.buildServer.buildTriggers.vcs.VcsChangesLoader.loadChangesForRoot(VcsChangesLoader.java:156)
at jetbrains.buildServer.vcs.impl.VcsManagerImpl.loadChanges(VcsManagerImpl.java:76)
at jetbrains.buildServer.serverSide.impl.auth.SecuredVcsManager.loadChanges(SecuredVcsManager.java:11)
at jetbrains.buildServer.serverSide.impl.VcsModificationChecker$1.run(VcsModificationChecker.java:17)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)

5 comments
Comment actions Permalink

After reading a post from 2009 about similar sounding issues with Star Team, I decided to detach the VCS roots from our projects, delete the VCS root, recreate it, and reattach it.  That seems to have magically fixed the problem, but I don't relish having to do this regularly.  We're going to upgrade to TeamCity 6.5.2 tonight to see if that helps at all as well.

0
Comment actions Permalink

I am having a very similar issue and am hoping to revive this thread.  Ours is not in error collection, but it is a VCS root that is failing when it's not been modified.  The VCS root is up and working.  After a day or so, it starts failing.  Here is some background:

- We are using Digest Authentication for the https:// hosted subversion
- We have a valid ssl cert which has not changed
- We have not changed the username or password in the VCS root at the time when the fetch starts failing (but did after for troubleshooting)
- Other VCS roots use the same username:password and continue to work fine
- If we change the VCS root to use another username:password, it works for a while, and then fails again.
- Switching it back to the original username:password works for a while, then the next day, it has been known to fail again
- We are running "TeamCity Professional 6.5.4 (build 18046)" on a Linux server connecting to a Linux svn server, served up by apache
- Nothing of note appears in the error_log in the svn server at the time of the issue
- I have updated the CATALINA_OPTS in teamcity-server.sh to include -Djava.awt.headless=true -Dsvnkit.http.methods=Basic,Digest,Negotiate,NTLM and restarted, to no avail

I am hoping not to have to add "ping-pong the user creds between two accounts" to my daily task list.  Does anyone have any suggestions for troubleshooting steps?  Thanks in advance for any help you can provide

Here is the whole build log, for reference (please pardon the size, just trying to be complete):

[14:43:42]: Checking for changes
[14:43:43]: Clean build enabled: removing old files from /home/production/Devel/project
[14:43:43]: Clearing temporary directory: /home/production/TeamCity/buildAgent/temp/buildTmp
[14:43:43]: Checkout directory: /home/production/Devel/project


[14:43:58]: [Updating sources: server side checkout...] Failed to build patch for build #644 {build id=1187}, VCS root: svn: https://our.server/svnroot/project/ {instance id=14, parent id=6}, due to error: org.tmatesoft.svn.core.SVNAuthenticationException: svn: Authentication required for '<https://our.server:443> Subversion Realm'
[14:43:58]: [Updating sources: server side checkout...] Repository sources transferred: 271.43Mb total
[14:43:58]: [Updating sources: server side checkout...] Average transfer speed: 19.18Mb per second
[14:43:58]: Will repeat attempt when server will be available, number of attempts left: 2
[14:44:08]: Updating sources: server side checkout... (14s)
[14:44:08]: [Updating sources: server side checkout...] Will perform clean checkout
[14:44:08]: [Updating sources: server side checkout...] Clean checkout reasons
[14:44:08]: [Clean checkout reasons] Checkout directory is empty or doesn't exist
[14:44:08]: [Clean checkout reasons] "Clean all files before build" turned on
[14:44:08]: [Updating sources: server side checkout...] Transferring cached clean patch for VCS root: project
[14:44:21]: [Updating sources: server side checkout...] Building incremental patch over the cached patch
[14:44:22]: [Updating sources: server side checkout...] Failed to build patch for build #644 {build id=1187}, VCS root: svn: https://our.server/svnroot/project/ {instance id=14, parent id=6}, due to error: org.tmatesoft.svn.core.SVNAuthenticationException: svn: Authentication required for '<https://our.server:443> Subversion Realm'
[14:44:22]: [Updating sources: server side checkout...] Repository sources transferred: 271.43Mb total
[14:44:22]: [Updating sources: server side checkout...] Average transfer speed: 19.41Mb per second
[14:44:22]: Will repeat attempt when server will be available, number of attempts left: 1
[14:44:32]: Updating sources: server side checkout... (12s)
[14:44:32]: [Updating sources: server side checkout...] Will perform clean checkout
[14:44:32]: [Updating sources: server side checkout...] Clean checkout reasons
[14:44:32]: [Clean checkout reasons] Checkout directory is empty or doesn't exist
[14:44:32]: [Clean checkout reasons] "Clean all files before build" turned on
[14:44:32]: [Updating sources: server side checkout...] Transferring cached clean patch for VCS root: project
[14:44:43]: [Updating sources: server side checkout...] Building incremental patch over the cached patch
[14:44:44]: [Updating sources: server side checkout...] Failed to build patch for build #644 {build id=1187}, VCS root: svn: https://our.server/svnroot/project/ {instance id=14, parent id=6}, due to error: org.tmatesoft.svn.core.SVNAuthenticationException: svn: Authentication required for '<https://our.server:443> Subversion Realm'
[14:44:44]: [Updating sources: server side checkout...] Repository sources transferred: 271.43Mb total
[14:44:44]: [Updating sources: server side checkout...] Average transfer speed: 22.64Mb per second
[14:44:44]: Patch is broken, can be found in file: /home/production/TeamCity/buildAgent/temp/cache/temp6014165479293080747patch_1187
[14:44:44]: Failed to build patch for build #644 {build id=1187}, VCS root: svn: https://our.server/svnroot/project/ {instance id=14, parent id=6}, due to error: org.tmatesoft.svn.core.SVNAuthenticationException: svn: Authentication required for '<https://our.server:443> Subversion Realm' jetbrains.buildServer.agent.impl.patch.PatchDownloaderImpl$1: Server was not able to build correct patch, most likely due to VCS errors, will try again.      at jetbrains.buildServer.agent.impl.patch.PatchDownloaderImpl.throwError(PatchDownloaderImpl.java:113)      at jetbrains.buildServer.agent.impl.patch.PatchDownloaderImpl.checkPatch(PatchDownloaderImpl.java:103)      at jetbrains.buildServer.agent.impl.patch.PatchDownloaderImpl.copyPatchAndCheck(PatchDownloaderImpl.java:64)      at jetbrains.buildServer.agent.impl.patch.UpdateSourcesPatcherBase.copyPatchToTempFile(UpdateSourcesPatcherBase.java:70)      at jetbrains.buildServer.agent.impl.patch.UpdateSourcesFromServer.updateSources(UpdateSourcesFromServer.java:62)      at jetbrains.buildServer.agent.impl.patch.UpdateSourcesBuildStageBase.doSourceUpdate(UpdateSourcesBuildStageBase.java:91)      at jetbrains.buildServer.agent.impl.patch.UpdateSourcesBuildStageBase.doRecoverableStage(UpdateSourcesBuildStageBase.java:59)      at jetbrains.buildServer.agent.impl.buildStages.startStages.RecoverableBuildStage.doLastAttempt(RecoverableBuildStage.java:112)      at jetbrains.buildServer.agent.impl.buildStages.startStages.RecoverableBuildStage.doBuildStage(RecoverableBuildStage.java:70)      at jetbrains.buildServer.agent.impl.buildStages.BuildStagesExecutor$1.callStage(BuildStagesExecutor.java:31)      at jetbrains.buildServer.agent.impl.buildStages.BuildStagesExecutor$1.callStage(BuildStagesExecutor.java:29)      at jetbrains.buildServer.agent.impl.buildStages.StagesExecutor.callRunStage(StagesExecutor.java:76)      at jetbrains.buildServer.agent.impl.buildStages.StagesExecutor.doStages(StagesExecutor.java:36)      at jetbrains.buildServer.agent.impl.buildStages.BuildStagesExecutor.doStages(BuildStagesExecutor.java:24)      at jetbrains.buildServer.agent.impl.BuildRunAction.doStages(BuildRunAction.java:65)      at jetbrains.buildServer.agent.impl.BuildRunAction.runBuild(BuildRunAction.java:52)      at jetbrains.buildServer.agent.impl.BuildAgentImpl.doActualBuild(BuildAgentImpl.java:245)      at jetbrains.buildServer.agent.impl.BuildAgentImpl.access$100(BuildAgentImpl.java:50)      at jetbrains.buildServer.agent.impl.BuildAgentImpl$1.run(BuildAgentImpl.java:218)      at java.lang.Thread.run(Thread.java:662) Caused by: jetbrains.buildServer.vcs.patches.UnsuccessfulPatchException: Failed to build patch for build #644 {build id=1187}, VCS root: svn: https://our.server/svnroot/project/ {instance id=14, parent id=6}, due to error: org.tmatesoft.svn.core.SVNAuthenticationException: svn: Authentication required for '<https://our.server:443> Subversion Realm'      at jetbrains.buildServer.vcs.patches.AbstractPatcher$1.fail(AbstractPatcher.java:93)      at jetbrains.buildServer.vcs.patches.LowLevelPatcher.readPatchStream(LowLevelPatcher.java:156)      at jetbrains.buildServer.vcs.patches.LowLevelPatcher.applyPatch(LowLevelPatcher.java:79)      at jetbrains.buildServer.vcs.patches.AbstractPatcher.applyPatch(AbstractPatcher.java:42)      at jetbrains.buildServer.agent.impl.patch.PatchApplierImpl.applyPatch(PatchApplierImpl.java:18)      at jetbrains.buildServer.agent.impl.patch.PatchDownloaderImpl.checkPatchInFileIsCompleted(PatchDownloaderImpl.java:83)      at jetbrains.buildServer.agent.impl.patch.PatchDownloaderImpl.checkPatch(PatchDownloaderImpl.java:93)      ... 18 more
0
Comment actions Permalink

Hello Scott,

  Sorry for delay with the answer.

  Could you please make sure that Keep-Alive connections are enabled in the apache settings of your server?

  TeamCity uses SVNKit library, and the version of the library provided with the latest TeamCity has a bug when
  Digest authentication may not work with non-Keep-Alive servers.

  Another thing to try - use Basic Authentication for your server, it is much more stable than Digest.

  Hope this helps,
  KIR

0
Comment actions Permalink

Hey Kirill,

No apologies.  This is free support on a community forum.  As far as I'm concerned, your response is timely and appreciated.

I have now ensured that KeepAlive is on at the server, however I expect is was before, as it is by default and there was nodirective disabling it.

Regarding Digest vs Basic:  Our group has opted for digest on our apache servers, as it is considered more secure.  If our problem returns or persists, we'll certainly consider reverting to basic as a troubleshooting step to see if it alleviates the issue.

As I mentioned in my original post, simply changing the user:pass in the SVN root did the job in the short term, but was getting a bit tiresome.  Since my last post, I have completely recreated the VCS root for the project (like the OP had done) and the connection has remained stable for one week.  I'm crossing my fingers that it's all good now.

Thanks again for your suggestions.

regards,
scott

0
Comment actions Permalink

Hello Scott,

  I hope that it will work.

  But if not, I suppose you may problems similar to those we have for NTLM authentication with SVN, which is also rather unstable.
  We have a note about this in our docs: http://confluence.jetbrains.net/display/TCD65/Known+Issues#KnownIssues-SubversionRepositoriesWithNTLMAuthorization

  One of possible tricks, mentioned there, is not to use domain account for authentication (and use local machine account).

  Good luck,
  KIR

0

Please sign in to leave a comment.