out of memory using git
I have some large files (generally a few megabytes, but sometimes a few hundred) that I don't expect to change often, but for ease of clean build would like to have under source control. I'm putting these under a separte VCS root for the TeamCity build, but can't get the checkout to work. I get the following errors (copied from the TeamCity webpage so the formatting may look a little strange).
Unable to collect changes: jetbrains.buildServer.vcs.VcsException: 'git fetch' command failed.
Show stacktrace »
Unable to collect changes: jetbrains.buildServer.vcs.VcsException: 'git fetch' command failed.
Show stacktrace »
java.util.concurrent.ExecutionException: jetbrains.buildServer.vcs.VcsException: 'git fetch' command failed.
stderr: Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at org.eclipse.jgit.transport.IndexPack.inflateAndReturn(IndexPack.java:959)
at org.eclipse.jgit.transport.IndexPack.whole(IndexPack.java:842)
at org.eclipse.jgit.transport.IndexPack.indexOneObject(IndexPack.java:801)
at org.eclipse.jgit.transport.IndexPack.index(IndexPack.java:404)
at org.eclipse.jgit.transport.BasePackFetchConnection.receivePack(BasePackFetchConnection.java:647)
at org.eclipse.jgit.transport.BasePackFetchConnection.doFetch(BasePackFetchConnection.java:279)
at org.eclipse.jgit.transport.BasePackFetchConnection.fetch(BasePackFetchConnection.java:224)
at org.eclipse.jgit.transport.FetchProcess.fetchObjects(FetchProcess.java:210)
at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:145)
at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:111)
at org.eclipse.jgit.transport.Transport.fetch(Transport.java:880)
at jetbrains.buildServer.buildTriggers.vcs.git.Fetcher.fetch(Fetcher.java:67)
at jetbrains.buildServer.buildTriggers.vcs.git.Fetcher.main(Fetcher.java:43)
at java.util.concurrent.FutureTask$Sync.innerGet(Unknown Source)
at java.util.concurrent.FutureTask.get(Unknown Source)
at jetbrains.buildServer.serverSide.impl.CancelableTaskHolder.waitForTaskToComplete(CancelableTaskHolder.java:29)
at jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl.doCollectChanges(TopBuildDependencyGraphImpl.java:99)
at jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl.access$000(TopBuildDependencyGraphImpl.java:21)
at jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl$3.run(TopBuildDependencyGraphImpl.java:2)
at jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl$3.run(TopBuildDependencyGraphImpl.java:1)
at jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl$4.run(TopBuildDependencyGraphImpl.java:1)
at jetbrains.buildServer.serverSide.impl.auth.SecurityContextImpl.runAs(SecurityContextImpl.java:1)
at jetbrains.buildServer.serverSide.impl.auth.SecurityContextImpl.runAsSystem(SecurityContextImpl.java:6)
at jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl.runAsSystem(TopBuildDependencyGraphImpl.java:90)
at jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl.collectChangesForGraph(TopBuildDependencyGraphImpl.java:19)
at jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl.collectChangesForGraph(TopBuildDependencyGraphImpl.java:47)
at jetbrains.buildServer.serverSide.impl.BuildChangesCollector.collectChangesForBuild(BuildChangesCollector.java:14)
at jetbrains.buildServer.serverSide.impl.BuildStarter$1.run(BuildStarter.java:14)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: jetbrains.buildServer.vcs.VcsException: 'git fetch' command failed.
stderr: Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at org.eclipse.jgit.transport.IndexPack.inflateAndReturn(IndexPack.java:959)
at org.eclipse.jgit.transport.IndexPack.whole(IndexPack.java:842)
at org.eclipse.jgit.transport.IndexPack.indexOneObject(IndexPack.java:801)
at org.eclipse.jgit.transport.IndexPack.index(IndexPack.java:404)
at org.eclipse.jgit.transport.BasePackFetchConnection.receivePack(BasePackFetchConnection.java:647)
at org.eclipse.jgit.transport.BasePackFetchConnection.doFetch(BasePackFetchConnection.java:279)
at org.eclipse.jgit.transport.BasePackFetchConnection.fetch(BasePackFetchConnection.java:224)
at org.eclipse.jgit.transport.FetchProcess.fetchObjects(FetchProcess.java:210)
at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:145)
at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:111)
at org.eclipse.jgit.transport.Transport.fetch(Transport.java:880)
at jetbrains.buildServer.buildTriggers.vcs.git.Fetcher.fetch(Fetcher.java:67)
at jetbrains.buildServer.buildTriggers.vcs.git.Fetcher.main(Fetcher.java:43)
at jetbrains.buildServer.buildTriggers.vcs.git.CommandLineUtil.getCommandLineError(CommandLineUtil.java:41)
at jetbrains.buildServer.buildTriggers.vcs.git.GitVcsSupport.fetchBranchDataInSeparateProcess(GitVcsSupport.java:1124)
at jetbrains.buildServer.buildTriggers.vcs.git.GitVcsSupport.fetch(GitVcsSupport.java:1017)
at jetbrains.buildServer.buildTriggers.vcs.git.GitVcsSupport.fetchBranchData(GitVcsSupport.java:1000)
at jetbrains.buildServer.buildTriggers.vcs.git.GitVcsSupport.getCurrentVersion(GitVcsSupport.java:946)
at jetbrains.buildServer.buildTriggers.vcs.VcsChangesLoader.getCurrentVersion(VcsChangesLoader.java:174)
at jetbrains.buildServer.vcs.impl.VcsManagerImpl.getVersionsForAllRoots(VcsManagerImpl.java:530)
at jetbrains.buildServer.vcs.impl.VcsManagerImpl.loadChanges(VcsManagerImpl.java:664)
at jetbrains.buildServer.serverSide.impl.auth.SecuredVcsManager.loadChanges(SecuredVcsManager.java:46)
at jetbrains.buildServer.vcs.impl.VcsChangesCollectorImpl$3.call(VcsChangesCollectorImpl.java:1)
at jetbrains.buildServer.vcs.impl.VcsChangesCollectorImpl$3.call(VcsChangesCollectorImpl.java:3)
... 5 more
stderr: Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at org.eclipse.jgit.transport.IndexPack.inflateAndReturn(IndexPack.java:959)
at org.eclipse.jgit.transport.IndexPack.whole(IndexPack.java:842)
at org.eclipse.jgit.transport.IndexPack.indexOneObject(IndexPack.java:801)
at org.eclipse.jgit.transport.IndexPack.index(IndexPack.java:404)
at org.eclipse.jgit.transport.BasePackFetchConnection.receivePack(BasePackFetchConnection.java:647)
at org.eclipse.jgit.transport.BasePackFetchConnection.doFetch(BasePackFetchConnection.java:279)
at org.eclipse.jgit.transport.BasePackFetchConnection.fetch(BasePackFetchConnection.java:224)
at org.eclipse.jgit.transport.FetchProcess.fetchObjects(FetchProcess.java:210)
at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:145)
at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:111)
at org.eclipse.jgit.transport.Transport.fetch(Transport.java:880)
at jetbrains.buildServer.buildTriggers.vcs.git.Fetcher.fetch(Fetcher.java:67)
at jetbrains.buildServer.buildTriggers.vcs.git.Fetcher.main(Fetcher.java:43)
at java.util.concurrent.FutureTask$Sync.innerGet(Unknown Source)
at java.util.concurrent.FutureTask.get(Unknown Source)
at jetbrains.buildServer.serverSide.impl.CancelableTaskHolder.waitForTaskToComplete(CancelableTaskHolder.java:29)
at jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl.doCollectChanges(TopBuildDependencyGraphImpl.java:99)
at jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl.access$000(TopBuildDependencyGraphImpl.java:21)
at jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl$3.run(TopBuildDependencyGraphImpl.java:2)
at jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl$3.run(TopBuildDependencyGraphImpl.java:1)
at jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl$4.run(TopBuildDependencyGraphImpl.java:1)
at jetbrains.buildServer.serverSide.impl.auth.SecurityContextImpl.runAs(SecurityContextImpl.java:1)
at jetbrains.buildServer.serverSide.impl.auth.SecurityContextImpl.runAsSystem(SecurityContextImpl.java:6)
at jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl.runAsSystem(TopBuildDependencyGraphImpl.java:90)
at jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl.collectChangesForGraph(TopBuildDependencyGraphImpl.java:19)
at jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl.collectChangesForGraph(TopBuildDependencyGraphImpl.java:47)
at jetbrains.buildServer.serverSide.impl.BuildChangesCollector.collectChangesForBuild(BuildChangesCollector.java:14)
at jetbrains.buildServer.serverSide.impl.BuildStarter$1.run(BuildStarter.java:14)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: jetbrains.buildServer.vcs.VcsException: 'git fetch' command failed.
stderr: Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at org.eclipse.jgit.transport.IndexPack.inflateAndReturn(IndexPack.java:959)
at org.eclipse.jgit.transport.IndexPack.whole(IndexPack.java:842)
at org.eclipse.jgit.transport.IndexPack.indexOneObject(IndexPack.java:801)
at org.eclipse.jgit.transport.IndexPack.index(IndexPack.java:404)
at org.eclipse.jgit.transport.BasePackFetchConnection.receivePack(BasePackFetchConnection.java:647)
at org.eclipse.jgit.transport.BasePackFetchConnection.doFetch(BasePackFetchConnection.java:279)
at org.eclipse.jgit.transport.BasePackFetchConnection.fetch(BasePackFetchConnection.java:224)
at org.eclipse.jgit.transport.FetchProcess.fetchObjects(FetchProcess.java:210)
at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:145)
at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:111)
at org.eclipse.jgit.transport.Transport.fetch(Transport.java:880)
at jetbrains.buildServer.buildTriggers.vcs.git.Fetcher.fetch(Fetcher.java:67)
at jetbrains.buildServer.buildTriggers.vcs.git.Fetcher.main(Fetcher.java:43)
at jetbrains.buildServer.buildTriggers.vcs.git.CommandLineUtil.getCommandLineError(CommandLineUtil.java:41)
at jetbrains.buildServer.buildTriggers.vcs.git.GitVcsSupport.fetchBranchDataInSeparateProcess(GitVcsSupport.java:1124)
at jetbrains.buildServer.buildTriggers.vcs.git.GitVcsSupport.fetch(GitVcsSupport.java:1017)
at jetbrains.buildServer.buildTriggers.vcs.git.GitVcsSupport.fetchBranchData(GitVcsSupport.java:1000)
at jetbrains.buildServer.buildTriggers.vcs.git.GitVcsSupport.getCurrentVersion(GitVcsSupport.java:946)
at jetbrains.buildServer.buildTriggers.vcs.VcsChangesLoader.getCurrentVersion(VcsChangesLoader.java:174)
at jetbrains.buildServer.vcs.impl.VcsManagerImpl.getVersionsForAllRoots(VcsManagerImpl.java:530)
at jetbrains.buildServer.vcs.impl.VcsManagerImpl.loadChanges(VcsManagerImpl.java:664)
at jetbrains.buildServer.serverSide.impl.auth.SecuredVcsManager.loadChanges(SecuredVcsManager.java:46)
at jetbrains.buildServer.vcs.impl.VcsChangesCollectorImpl$3.call(VcsChangesCollectorImpl.java:1)
at jetbrains.buildServer.vcs.impl.VcsChangesCollectorImpl$3.call(VcsChangesCollectorImpl.java:3)
... 5 more
Edit this VCS root » VCS problem: Big Data
'git fetch' command failed.
stderr: Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at org.eclipse.jgit.transport.IndexPack.inflateAndReturn(IndexPack.java:959)
at org.eclipse.jgit.transport.IndexPack.whole(IndexPack.java:842)
at org.eclipse.jgit.transport.IndexPack.indexOneObject(IndexPack.java:801)
at org.eclipse.jgit.transport.IndexPack.index(IndexPack.java:404)
at org.eclipse.jgit.transport.BasePackFetchConnection.receivePack(BasePackFetchConnection.java:647)
at org.eclipse.jgit.transport.BasePackFetchConnection.doFetch(BasePackFetchConnection.java:279)
at org.eclipse.jgit.transport.BasePackFetchConnection.fetch(BasePackFetchConnection.java:224)
at org.eclipse.jgit.transport.FetchProcess.fetchObjects(FetchProcess.java:210)
at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:145)
at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:111)
at org.eclipse.jgit.transport.Transport.fetch(Transport.java:880)
at jetbrains.buildServer.buildTriggers.vcs.git.Fetcher.fetch(Fetcher.java:67)
at jetbrains.buildServer.buildTriggers.vcs.git.Fetcher.main(Fetcher.java:43)
Show stacktrace »
Is there a setting available to give the git fetch more memory? Anything else I can try?
Please sign in to leave a comment.
Hello,
you can try to increase amount of memory for git clone operation by using 'teamcity.git.fetch.process.max.memory' internal property.
By default it is set to 512M. On the server we use jgit - java implementation of the git and it has problems with big objects and big repositories.
If increase of memory won't help you can clone your repository manually and tell TeamCity to use your clone.