Error collecting changes for VCS repository (Bonobo Git)
I have a build configuration attached to a VCS Git (Bonobo). When I created it everything went fine, I set it up to check out on the agent itself and everything worked. However after a while (without any commits by a developer) the connection to the git fails with an error message. After this error occured I got a message that it would not be possible to check out files on the agent.
The Repository is considerably large, however it never poses a problem when setting up a new build configuration.
Here the error message:
Error collecting changes for VCS repository '"switchANALYSIS Master" {instance id=25, parent internal id=5, parent id=SwitchANALYSISBeta_SwitchANALYSISMaster, description: "http://teamcity/bonobo/MetroAnalyzer.git#refs/heads/master"}'
'git fetch' command failed.
stderr: java.lang.OutOfMemoryError: Java heap space
at org.eclipse.jgit.internal.storage.pack.BinaryDelta.apply(BinaryDelta.java:163)
at org.eclipse.jgit.internal.storage.pack.BinaryDelta.apply(BinaryDelta.java:118)
at org.eclipse.jgit.transport.PackParser.resolveDeltas(PackParser.java:610)
at org.eclipse.jgit.transport.PackParser.resolveDeltas(PackParser.java:587)
at org.eclipse.jgit.transport.PackParser.resolveDeltas(PackParser.java:550)
at org.eclipse.jgit.transport.PackParser.parse(PackParser.java:507)
at org.eclipse.jgit.internal.storage.file.ObjectDirectoryPackParser.parse(ObjectDirectoryPackParser.java:194)
at org.eclipse.jgit.transport.PackParser.parse(PackParser.java:448)
at org.eclipse.jgit.transport.BasePackFetchConnection.receivePack(BasePackFetchConnection.java:762)
at org.eclipse.jgit.transport.BasePackFetchConnection.doFetch(BasePackFetchConnection.java:363)
at org.eclipse.jgit.transport.TransportHttp$SmartHttpFetchConnection.doFetch(TransportHttp.java:789)
at org.eclipse.jgit.transport.BasePackFetchConnection.fetch(BasePackFetchConnection.java:301)
at org.eclipse.jgit.transport.BasePackFetchConnection.fetch(BasePackFetchConnection.java:291)
at org.eclipse.jgit.transport.FetchProcess.fetchObjects(FetchProcess.java:247)
at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:160)
at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:122)
at org.eclipse.jgit.transport.Transport.fetch(Transport.java:1138)
at jetbrains.buildServer.buildTriggers.vcs.git.GitServerUtil.fetch(GitServerUtil.java:511)
at jetbrains.buildServer.buildTriggers.vcs.git.Fetcher.fetch(Fetcher.java:112)
at jetbrains.buildServer.buildTriggers.vcs.git.Fetcher.main(Fetcher.java:64)
stdout: remote: Counting objects: 87
remote: Counting objects: 3199
remote: Counting objects: 4654
remote: Counting objects: 5311
remote: Counting objects: 6151
remote: Counting objects: 7306
remote: Counting objects: 8274
remote: Counting objects: 8537
remote: Counting objects: 9925
remote: Counting objects: 10071
remote: Counting objects: 10187
remote: Counting objects: 10310
remote: Counting objects: 10461
remote: Counting objects: 10617
remote: Counting objects: 10745
remote: Counting objects: 12987
remote: Counting objects: 18150
remote: Counting objects: 43649
remote: Counting objects: 51326
remote: Compressing objects: 1%
remote: Compressing objects: 2%
remote: Compressing objects: 3%
remote: Compressing objects: 4%
remote: Compressing objects: 5%
remote: Compressing objects: 6%
remote: Compressing objects: 7%
remote: Compressing objects: 8%
remote: Compressing objects: 9%
remote: Compressing objects: 10%
remote: Compressing objects: 11%
remote: Compressing objects: 12%
remote: Compressing objects: 13%
remote: Compressing objects: 14%
remote: Compressing objects: 15%
remote: Compressing objects: 16%
remote: Compressing objects: 17%
remote: Compressing objects: 18%
remote: Compressing objects: 19%
remote: Compressing objects: 20%
remote: Compressing objects: 21%
remote: Compressing objects: 22%
remote: Compressing objects: 23%
remote: Compressing objects: 24%
remote: Compressing objects: 25%
remote: Compressing objects: 26%
remote: Compressing objects: 27%
remote: Compressing objects: 28%
remote: Compressing objects: 29%
remote: Compressing objects: 30%
remote: Compressing objects: 31%
remote: Compressing objects: 32%
remote: Compressing objects: 33%
remote: Compressing objects: 34%
remote: Compressing objects: 35%
remote: Compressing objects: 36%
remote: Compressing objects: 37%
remote: Compressing objects: 38%
remote: Compressing objects: 39%
remote: Compressing objects: 40%
remote: Compressing objects: 41%
remote: Compressing objects: 42%
remote: Compressing objects: 43%
remote: Compressing objects: 44%
remote: Compressing objects: 45%
remote: Compressing objects: 46%
remote: Compressing objects: 47%
remote: Compressing objects: 48%
remote: Compressing objects: 49%
remote: Compressing objects: 50%
remote: Compressing objects: 51%
remote: Compressing objects: 52%
remote: Compressing objects: 53%
remote: Compressing objects: 54%
remote: Compressing objects: 55%
remote: Compressing objects: 56%
remote: Compressing objects: 57%
remote: Compressing objects: 58%
remote: Compressing objects: 59%
remote: Compressing objects: 60%
remote: Compressing objects: 61%
remote: Compressing objects: 62%
remote: Compressing objects: 63%
remote: Compressing objects: 64%
remote: Compressing objects: 65%
remote: Compressing objects: 66%
remote: Compressing objects: 67%
remote: Compressing objects: 68%
remote: Compressing objects: 69%
remote: Compressing objects: 70%
remote: Compressing objects: 71%
remote: Compressing objects: 72%
remote: Compressing objects: 73%
remote: Compressing objects: 74%
remote: Compressing objects: 75%
remote: Compressing objects: 76%
remote: Compressing objects: 77%
remote: Compressing objects: 78%
remote: Compressing objects: 79%
remote: Compressing objects: 80%
remote: Compressing objects: 81%
remote: Compressing objects: 82%
remote: Compressing objects: 83%
remote: Compressing objects: 84%
remote: Compressing objects: 85%
remote: Compressing objects: 86%
remote: Compressing objects: 87%
remote: Compressing objects: 88%
remote: Compressing objects: 89%
remote: Compressing objects: 90%
remote: Compressing objects: 91%
remote: Compressing objects: 92%
remote: Compressing objects: 93%
remote: Compressing objects: 94%
remote: Compressing objects: 95%
remote: Compressing objects: 96%
remote: Compressing objects: 97%
remote: Compressing objects: 98%
remote: Compressing objects: 99%
remote: Compressing objects: 100%
remote: Compressing objects: 100%
Receiving objects: 1%
Receiving objects: 2%
Receiving objects: 3%
Receiving objects: 4%
Receiving objects: 5%
Receiving objects: 6%
Receiving objects: 7%
Receiving objects: 8%
Receiving objects: 9%
Receiving objects: 10%
Receiving objects: 11%
Receiving objects: 12%
Receiving objects: 13%
Receiving objects: 14%
Receiving objects: 15%
Receiving objects: 16%
Receiving objects: 17%
Receiving objects: 18%
Receiving objects: 19%
Receiving objects: 20%
Receiving objects: 21%
Receiving objects: 22%
Receiving objects: 23%
Receiving objects: 24%
Receiving objects: 25%
Receiving objects: 26%
Receiving objects: 27%
Receiving objects: 28%
Receiving objects: 29%
Receiving objects: 30%
Receiving objects: 31%
Receiving objects: 32%
Receiving objects: 33%
Receiving objects: 34%
Receiving objects: 35%
Receiving objects: 36%
Receiving objects: 37%
Receiving objects: 38%
Receiving objects: 39%
Receiving objects: 40%
Receiving objects: 41%
Receiving objects: 42%
Receiving objects: 43%
Receiving objects: 44%
Receiving objects: 45%
Receiving objects: 46%
Receiving objects: 47%
Receiving objects: 48%
Receiving objects: 49%
Receiving objects: 50%
Receiving objects: 51%
Receiving objects: 52%
Receiving objects: 53%
Receiving objects: 54%
Receiving objects: 55%
Receiving objects: 56%
Receiving objects: 57%
Receiving objects: 58%
Receiving objects: 59%
Receiving objects: 60%
Receiving objects: 61%
Receiving objects: 62%
Receiving objects: 63%
Receiving objects: 64%
Receiving objects: 65%
Receiving objects: 66%
Receiving objects: 67%
Receiving objects: 68%
Receiving objects: 69%
Receiving objects: 70%
Receiving objects: 71%
Receiving objects: 72%
Receiving objects: 73%
Receiving objects: 74%
Receiving objects: 75%
Receiving objects: 76%
Receiving objects: 77%
Receiving objects: 78%
Receiving objects: 79%
Receiving objects: 80%
Receiving objects: 81%
Receiving objects: 82%
Receiving objects: 83%
Receiving objects: 84%
Receiving objects: 85%
Receiving objects: 86%
Receiving objects: 87%
Receiving objects: 88%
Receiving objects: 89%
Receiving objects: 90%
Receiving objects: 91%
Receiving objects: 92%
Receiving objects: 93%
Receiving objects: 94%
Receiving objects: 95%
Receiving objects: 96%
Receiving objects: 97%
Receiving objects: 98%
Receiving objects: 99%
Receiving objects: 100%
Receiving objects: 100%
Resolving deltas: 1%
Resolving deltas: 2%
Resolving deltas: 3%
Resolving deltas: 4%
Resolving deltas: 5%
Resolving deltas: 6%
Resolving deltas: 7%
Resolving deltas: 8%
Resolving deltas: 9%
Resolving deltas: 10%
Resolving deltas: 11%
Resolving deltas: 12%
Resolving deltas: 13%
Resolving deltas: 14%
Resolving deltas: 15%
Resolving deltas: 16%
Resolving deltas: 17%
Resolving deltas: 18%
Resolving deltas: 19%
Resolving deltas: 20%
Resolving deltas: 21%
Resolving deltas: 22%
Resolving deltas: 23%
Resolving deltas: 24%
Resolving deltas: 25%
Resolving deltas: 26%
Resolving deltas: 27%
Resolving deltas: 28%
Resolving deltas: 29%
Resolving deltas: 30%
Resolving deltas: 31%
Resolving deltas: 32%
Resolving deltas: 33%
Resolving deltas: 34%
Resolving deltas: 35%
Resolving deltas: 36%
Resolving deltas: 37%
Resolving deltas: 38%
Resolving deltas: 39%
Resolving deltas: 40%
Resolving deltas: 41%
Resolving deltas: 42%
Resolving deltas: 43%
Resolving deltas: 44%
Resolving deltas: 45%
Resolving deltas: 46%
Resolving deltas: 47%
Resolving deltas: 48%
Resolving deltas: 49%
Resolving deltas: 50%
Resolving deltas: 51%
Resolving deltas: 52%
exit code: 1
Please sign in to leave a comment.
Hi Felix,
If your repository is large enough, trying to load too much information into memory might lead to issues. The git fetch is performed in a separate process, so you might need to consider increasing the allowed memory for that process. In particular, you can use the "teamcity.git.fetch.process.max.memory" parameter and increase it, as described here: https://confluence.jetbrains.com/display/TCD10/Git#Git-InternalProperties
In the same article, if you aren't using the last version of teamcity, consider enabling the git garbage collection on the server.
Hi Denis,
thank you very much for your answer. I added "teamcity.git.fetch.process.max.memory=1024M" to the internal properties and restarted the server. So far I still have the same result.
Some interesting details I can not make sense of so far are:
Could the Git itself have a problem? Could it be the fact that the Git Repository was changed?
I have a Biuld of a different project running as well and this project is able to collect the changes without any problems.
Any Ideas?
Please check if increasing it to 2048M helps.
Hallo Dmitry Neverov,
Thank you very much for your tip, I increased the "teamcity.git.fetch.process.max.memory" to 2048M. This changed the error message:
Failed to collect changes...
'git fetch' command failed. stdout: Error occurred during initialization of VM Could not reserve enough space for 2097152KB object heap exit code: 1
Any idea why git would try to reserve 209GB? Is this the repository size???
I do not understand why a change in the repository leads to this kind of problem, but a fresh build (newly configured build and vcs connection) does not have a problem like this.
2097152Kb из 2048Mb. Most likely it fails because you use 32-bit java. Try to set the limit to 1200M, 32-bit java should be able to allocate this amount of memory. If checkout still fails, then you need to install 64-bit java (https://confluence.jetbrains.com/display/TCD10/Installing+and+Configuring+the+TeamCity+Server#InstallingandConfiguringtheTeamCityServer-x64jvm), and set the limit to 2048M.
When a build configuration is first created and build is started TeamCity doesn't detect any changes (since there were no commits since the build configuration creation) and build starts fine. Once commits are detected TeamCity needs to clone the git repository to provide information about commits in UI and fails to do so.
Hi Dmitry,
The 64-bit java did the trick. Thank you so much, you are the best.