SVN Update generating Locked Repository Error

I just did an in-place upgrade to 2008 Server 64bit and updated Java to V6U18 and now every other build failure when Team City does an update of the local repository. Prior to the upgrade, we were running 2008 64bit, only it was the RC version.

The version of TeamCity I am using is 4.5.4 (build 9071).

Prior to the upgrade, there was never an issue updating the local repository. Now every other build fails reliably with the below error. Settings on the VCS Root are as follows:
  Checkout Rules: None
  Checkout Mode: Automatically on Agent
  Externals: Checkout but ignore changes

Here is the output of the build error (servers and paths have been altered):


    [04:07:01]: [Updating sources]
    jetbrains.buildServer.vcs.VcsException: org.tmatesoft.svn.core.SVNException: svn: Error during update from https://svn/CTS_ProjectA/trunk to H:\build\ProjectA\trunk
    svn: Working copy 'H:\build\ProjectA' locked; try performing 'cleanup'
    jetbrains.buildServer.vcs.VcsException: org.tmatesoft.svn.core.SVNException: svn: Error during update from https://svn/CTS_ProjectA/trunk to H:\build\ProjectA\trunk
    svn: Working copy 'H:\build\ProjectA' locked; try performing 'cleanup'
      at jetbrains.buildServer.vcs.svn.SvnCheckoutOnAgentSupport.updateSources(SvnCheckoutOnAgentSupport.java:104)
      at jetbrains.buildServer.agent.impl.vcs.CheckoutByIncludeRuleAdapter$1$1.process(CheckoutByIncludeRuleAdapter.java:28)
      at jetbrains.buildServer.agent.impl.vcs.CheckoutByIncludeRuleAdapter$2.updateSources(CheckoutByIncludeRuleAdapter.java:49)
      at jetbrains.buildServer.agent.impl.patch.ProjectSourcesOnAgent.checkoutSources(ProjectSourcesOnAgent.java:61)
      at jetbrains.buildServer.agent.impl.patch.ProjectSourcesGetter.checkoutSources(ProjectSourcesGetter.java:269)
      at jetbrains.buildServer.agent.impl.patch.ProjectSourcesGetter.execute(ProjectSourcesGetter.java:123)
      at jetbrains.buildServer.agent.impl.runStages.CheckoutSourcesStage.doGetSources(CheckoutSourcesStage.java:45)
      at jetbrains.buildServer.agent.impl.runStages.CheckoutSourcesStage.doRecoverableStage(CheckoutSourcesStage.java:31)
      at jetbrains.buildServer.agent.impl.runStages.RecoverableBuildStage.doBuildStage(RecoverableBuildStage.java:61)
      at jetbrains.buildServer.agent.impl.BuildRunAction.doStages(BuildRunAction.java:135)
      at jetbrains.buildServer.agent.impl.BuildRunAction.access$000(BuildRunAction.java:21)
      at jetbrains.buildServer.agent.impl.BuildRunAction$1.run(BuildRunAction.java:91)
      at java.lang.Thread.run(Thread.java:595)
    Caused by: org.tmatesoft.svn.core.SVNException: svn: Error during update from https://svn/CTS_ProjectA/trunk to H:\build\ProjectA\trunk
    svn: Working copy 'H:\build\ProjectA' locked; try performing 'cleanup'
      at jetbrains.buildServer.vcs.svn.TCSvnUpdateClient.doCheckout(TCSvnUpdateClient.java:82)
      at jetbrains.buildServer.vcs.svn.SvnCheckoutOnAgentSupport.updateSources(SvnCheckoutOnAgentSupport.java:73)
      ... 12 more
    Caused by: org.tmatesoft.svn.core.SVNException: svn: Working copy 'H:\build\ProjectA' locked; try performing 'cleanup'
      at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:64)
      at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:51)
      at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:612)
      at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:273)
      at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:261)
      at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.doReport(DAVConnection.java:266)
      at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.runReport(DAVRepository.java:1261)
      at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.update(DAVRepository.java:818)
      at org.tmatesoft.svn.core.wc.SVNUpdateClient.update(SVNUpdateClient.java:558)
      at org.tmatesoft.svn.core.wc.SVNUpdateClient.doCheckout(SVNUpdateClient.java:922)
      at jetbrains.buildServer.vcs.svn.TCSvnUpdateClient.doCheckout(TCSvnUpdateClient.java:79)
      ... 13 more

I have debug logging turned on for SVN and can post a file if needed of the 1000 or so lines from one of the failures.

Briefly, here is what looks to be the common error output often on this same directory though running manually svn cleanup or svn update on this directory causes no errors.

    [2010-03-05 07:01:36,037]  DEBUG -                 javasvn.output - NETWORK: SPOOLING RESPONSE TO FILE: C:\Program Files\TeamCity\buildAgent\temp\cache\svnkit.5755.spool
    [2010-03-05 07:01:36,037]  DEBUG -                 javasvn.output - NETWORK: READ
    <?xml version="1.0" encoding="utf-8"?>
    <S:update-report xmlns:S="svn:" xmlns:V="http://subversion.tigris.org/xmlns/dav/" xmlns:D="DAV:" send-all="true">
    <S:target-revision rev="5"/>
    <S:open-directory rev="5">
    <D:checked-in><D:href>/CTS_Logging/!svn/ver/5/tags/2009.1.1_2.0</D:href></D:checked-in>
    <S:set-prop name="svn:entry:committed-rev">5</S:set-prop>
    <S:set-prop name="svn:entry:committed-date">2009-10-02T12:00:20.665016Z</S:set-prop>
    <S:set-prop name="svn:entry:last-author">JUser</S:set-prop>
    <S:set-prop name="svn:entry:uuid">a5900edc-bef5-4158-ab9f-384324052309</S:set-prop>
    <S:prop></S:prop>
    </S:open-directory>
    </S:update-report>

    [2010-03-05 07:01:36,037]  DEBUG -                 javasvn.output - NETWORK: SPOOLED: 642
    [2010-03-05 07:01:36,037]  DEBUG -                 javasvn.output - NETWORK: READING SPOOLED FILE: C:\Program Files\TeamCity\buildAgent\temp\cache\svnkit.5755.spool
    [2010-03-05 07:01:36,037]  DEBUG -                 javasvn.output - NETWORK: ABOUT TO READ: 642
    [2010-03-05 07:01:36,037]  DEBUG -                 javasvn.output - NETWORK: SPOOLED RESPONSE FULLY READ
    [2010-03-05 07:01:36,037]  DEBUG -                 javasvn.output - NETWORK: FAILED TO READ SPOOLED RESPONSE FULLY (no more files): 642
    [2010-03-05 07:01:36,037]  DEBUG -                 javasvn.output - WC: root
    [2010-03-05 07:01:36,037]  DEBUG -                 javasvn.output - WC: change dir prop svn:wc:ra_dav:version-url = /CTS_Logging/!svn/ver/5/tags/2009.1.1_2.0
    [2010-03-05 07:01:36,053]  DEBUG -                 javasvn.output - WC: change dir prop svn:entry:committed-rev = 5
    [2010-03-05 07:01:36,053]  DEBUG -                 javasvn.output - WC: change dir prop svn:entry:committed-date = 2009-10-02T12:00:20.665016Z
    [2010-03-05 07:01:36,053]  DEBUG -                 javasvn.output - WC: change dir prop svn:entry:last-author = JUser
    [2010-03-05 07:01:36,053]  DEBUG -                 javasvn.output - WC: change dir prop svn:entry:uuid = a5900edc-bef5-4158-ab9f-384324052309
    [2010-03-05 07:01:36,053]  DEBUG -                 javasvn.output - WC: close dir
    [2010-03-05 07:01:36,053]  DEBUG -                 javasvn.output - DEFAULT: svn: Can't read length line from file H:\build\ProjectA\trunk\src\external\Logging\.svn\all-wcprops
    [2010-03-05 07:01:36,068]  DEBUG -                 javasvn.output - WC: close edit
    [2010-03-05 07:01:36,100]  DEBUG -                 javasvn.output - NETWORK: FAILED TO READ SPOOLED RESPONSE FULLY (no more files): -1

Any assistance on where to look next would be greatly appreciated.

Thanks,
Ken



Attachment(s):
SVN_Debug.log.zip
2 comments

Hello Ken,

  Have you tried to enforce clean checkout for the corresponding build configuration?
  You can do this via Build configuration page -> Actions -> Enforce clean checkout

  Please also make sure that directory H:\build\ProjectA\trunk is updated by TeamCity only and there are no other processes which work with this directory besides TeamCity.

  Usually the "Locked Repository" problem arises when either SVN command didn't finish correctly, or if there are multiple processes which change the same SVN working client directory.

  Please let me know if this helps,
  Regards,
  KIR

0

KIR,

Thanks for the tips.

I had seen the process do a clean checkout before, but the error had remained. This time, I forced a clean checkout via your described method and it caused an error when trying to checkout into the now empty directory.

What I found based on that error was we had recently switched to checking out at the Trunk and Branch level as opposed to the VCS root due to the fact tags were being brought down to the agent at every new tag, causing very slow checkouts and large disk usage. (I saw a post on this while searching for my answer and knew exactly what the person was referring to - we'd just experienced it and changed our method of checkout) What we failed to do when moving the checkout points was to remove the .svn directory from the project root directory. Therefore the root .svn configuration was not out of alignment with the remainder of the file system since we had been checking out against trunk and branches.

I deleted the project directory completely and recreated it, allowing TC to do a completely clean checkout and that appears to have solved our issues.

Thanks again for the response.

Ken

0

Please sign in to leave a comment.