TeamCity V6 - OutofMemoryError on 'checking changes'

I've tried a few different versions, but I cannot get git checkouts to work.  I get the following stacktrace during "checking changes".  I have tried increasing the memory in the properties file, but I don't notice a difference.

Caused by: jetbrains.buildServer.vcs.VcsException: 'git fetch' command failed.
stderr: java.lang.OutOfMemoryError: Java heap space
at org.eclipse.jgit.transport.IndexPack.inflateAndReturn(IndexPack.java:1033)
at org.eclipse.jgit.transport.IndexPack.resolveDeltas(IndexPack.java:518)
at org.eclipse.jgit.transport.IndexPack.resolveDeltas(IndexPack.java:495)
at org.eclipse.jgit.transport.IndexPack.resolveDeltas(IndexPack.java:484)
at org.eclipse.jgit.transport.IndexPack.index(IndexPack.java:424)
at org.eclipse.jgit.transport.BasePackFetchConnection.receivePack(BasePackFetchConnection.java:647)
at org.eclipse.jgit.transport.BasePackFetchConnection.doFetch(BasePackFetchConnection.java:279)
at org.eclipse.jgit.transport.BasePackFetchConnection.fetch(BasePackFetchConnection.java:224)
at org.eclipse.jgit.transport.FetchProcess.fetchObjects(FetchProcess.java:214)
at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:149)
at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:111)
at org.eclipse.jgit.transport.Transport.fetch(Transport.java:904)
at jetbrains.buildServer.buildTriggers.vcs.git.Fetcher.fetch(Fetcher.java:78)
at jetbrains.buildServer.buildTriggers.vcs.git.Fetcher.main(Fetcher.java:46)
at jetbrains.buildServer.buildTriggers.vcs.git.CommandLineUtil.getCommandLineError(CommandLineUtil.java:41)
at jetbrains.buildServer.buildTriggers.vcs.git.GitVcsSupport.fetchInSeparateProcess(GitVcsSupport.java:1114)
at jetbrains.buildServer.buildTriggers.vcs.git.GitVcsSupport.fetch(GitVcsSupport.java:1005)
at jetbrains.buildServer.buildTriggers.vcs.git.GitVcsSupport.fetchBranchData(GitVcsSupport.java:988)
at jetbrains.buildServer.buildTriggers.vcs.git.GitVcsSupport.getCurrentVersion(GitVcsSupport.java:931)
at jetbrains.buildServer.buildTriggers.vcs.VcsChangesLoader.getCurrentVersion(VcsChangesLoader.java:69)
at jetbrains.buildServer.vcs.impl.VcsManagerImpl.getVersionsForAllRoots(VcsManagerImpl.java:330)
at jetbrains.buildServer.vcs.impl.VcsManagerImpl.loadChanges(VcsManagerImpl.java:467)
at jetbrains.buildServer.serverSide.impl.auth.SecuredVcsManager.loadChanges(SecuredVcsManager.java:28)
at jetbrains.buildServer.vcs.impl.VcsChangesCollectorImpl$3.call(VcsChangesCollectorImpl.java:3)
at jetbrains.buildServer.vcs.impl.VcsChangesCollectorImpl$3.call(VcsChangesCollectorImpl.java)

13 comments
Comment actions Permalink

Hi Adam,
how much memory did you give to fetch process and how big is your repository?  
If you repository is quite big you may clone it using a native git and set up VCS Root to use it. In this case Teamcity will do just small fetches which should not cause OutOfMemoryErrors.

0
Comment actions Permalink

Where would I clone this to?  Cloning it to the agent work directory and naming that directory causes the directroy to be nuked.

0
Comment actions Permalink

Hi Raymond,

this error occurs only on the server, so if you get this error message you could clone your repository somewhere on the server and set a VCS root option "Clone repository to:" to the dir with the clone.
But I would first try to increase amount of memory for the fetch process using teamcity.git.fetch.process.max.memory internal option, please see http://confluence.jetbrains.net/display/TCD65/Git+%28JetBrains%29#Git%28JetBrains%29-InternalProperties for details.

0
Comment actions Permalink

I have already bumped up the settings to:

teamcity.git.fetch.process.max.memory=7168M
teamcity.git.fetch.separate.process=true

In the TeamCity/conf/internal.properties file with no luck.  This is where I would set these, correct?

0
Comment actions Permalink

Yes, this is the right place.
Is your repository so big? If so - try clone it manually and point to it in the settings of the VCS root.

0
Comment actions Permalink

The repo is around 300MB with about a billion commits over the last number of years.  Working on the alternate solution and will let you know how it goes.

Thanks!

0
Comment actions Permalink
I'm still having the issue.  Here's what I have:


Project is 'Core', branch and build are both called 'ray_testing'


'Clone repo to' in VCS root config is '/mnt/repos/core_ray_testing'


I cloned the repo to that directory and checked out the 'ray_testing' branch.



Memory settings are as mentioned before.  The server has 8GB of RAM.



...and then:







[11:22:32]: Checking for changes

[11:27:59]: Unable to collect changes java.util.concurrent.ExecutionException:  jetbrains.buildServer.vcs.VcsException: Problem collecting changes for  'M2 Core :: ray_testing' : 'git fetch' command failed. stderr: java.lang.OutOfMemoryError: Java heap space      at org.eclipse.jgit.storage.pack.BinaryDelta.apply(BinaryDelta.java:163)      at org.eclipse.jgit.storage.pack.BinaryDelta.apply(BinaryDelta.java:118)      at org.eclipse.jgit.transport.PackParser.resolveDeltas(PackParser.java:561)      at org.eclipse.jgit.transport.PackParser.resolveDeltas(PackParser.java:544)      at org.eclipse.jgit.transport.PackParser.resolveDeltas(PackParser.java:507)      at org.eclipse.jgit.transport.PackParser.parse(PackParser.java:462)      at org.eclipse.jgit.storage.file.ObjectDirectoryPackParser.parse(ObjectDirectoryPackParser.java:178)      at org.eclipse.jgit.transport.PackParser.parse(PackParser.java:410)      at org.eclipse.jgit.transport.BasePackFetchConnection.receivePack(BasePackFetchConnection.java:649)      at org.eclipse.jgit.transport.BasePackFetchConnection.doFetch(BasePackFetchConnection.java:280)      at org.eclipse.jgit.transport.BasePackFetchConnection.fetch(BasePackFetchConnection.java:225)      at org.eclipse.jgit.transport.FetchProcess.fetchObjects(FetchProcess.java:214)      at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:149)      at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:111)      at org.eclipse.jgit.transport.Transport.fetch(Transport.java:903)      at jetbrains.buildServer.buildTriggers.vcs.git.Fetcher.fetch(Fetcher.java:81)      at jetbrains.buildServer.buildTriggers.vcs.git.Fetcher.main(Fetcher.java:47)      at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)      at java.util.concurrent.FutureTask.get(FutureTask.java:83)      at jetbrains.buildServer.serverSide.impl.CancelableTaskHolder.waitForTaskToComplete(CancelableTaskHolder.java:1)      at  jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl.doCollectChanges(TopBuildDependencyGraphImpl.java:9)      at jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl.access$000(TopBuildDependencyGraphImpl.java:29)      at jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl$3.run(TopBuildDependencyGraphImpl.java:1)      at jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl$3.run(TopBuildDependencyGraphImpl.java)      at jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl$4.run(TopBuildDependencyGraphImpl.java:1)      at jetbrains.buildServer.serverSide.impl.auth.SecurityContextImpl.runAs(SecurityContextImpl.java:21)      at jetbrains.buildServer.serverSide.impl.auth.SecurityContextImpl.runAsSystem(SecurityContextImpl.java:1)      at  jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl.runAsSystem(TopBuildDependencyGraphImpl.java:66)      at  jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl.collectChangesForGraph(TopBuildDependencyGraphImpl.java:82)      at  jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl.collectChangesForGraph(TopBuildDependencyGraphImpl.java:27)      at jetbrains.buildServer.serverSide.impl.BuildChangesCollector.collectChangesForBuild(BuildChangesCollector.java:21)      at jetbrains.buildServer.serverSide.impl.BuildStarter$1.run(BuildStarter.java:18)      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)      at java.util.concurrent.FutureTask.run(FutureTask.java:138)      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)      at java.lang.Thread.run(Thread.java:619) Caused by: jetbrains.buildServer.vcs.VcsException: Problem collecting  changes for 'M2 Core :: ray_testing' : 'git fetch' command failed. stderr: java.lang.OutOfMemoryError: Java heap space      at org.eclipse.jgit.storage.pack.BinaryDelta.apply(BinaryDelta.java:163)      at org.eclipse.jgit.storage.pack.BinaryDelta.apply(BinaryDelta.java:118)      at org.eclipse.jgit.transport.PackParser.resolveDeltas(PackParser.java:561)      at org.eclipse.jgit.transport.PackParser.resolveDeltas(PackParser.java:544)      at org.eclipse.jgit.transport.PackParser.resolveDeltas(PackParser.java:507)      at org.eclipse.jgit.transport.PackParser.parse(PackParser.java:462)      at org.eclipse.jgit.storage.file.ObjectDirectoryPackParser.parse(ObjectDirectoryPackParser.java:178)      at org.eclipse.jgit.transport.PackParser.parse(PackParser.java:410)      at org.eclipse.jgit.transport.BasePackFetchConnection.receivePack(BasePackFetchConnection.java:649)      at org.eclipse.jgit.transport.BasePackFetchConnection.doFetch(BasePackFetchConnection.java:280)      at org.eclipse.jgit.transport.BasePackFetchConnection.fetch(BasePackFetchConnection.java:225)      at org.eclipse.jgit.transport.FetchProcess.fetchObjects(FetchProcess.java:214)      at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:149)      at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:111)      at org.eclipse.jgit.transport.Transport.fetch(Transport.java:903)      at jetbrains.buildServer.buildTriggers.vcs.git.Fetcher.fetch(Fetcher.java:81)      at jetbrains.buildServer.buildTriggers.vcs.git.Fetcher.main(Fetcher.java:47)      at  jetbrains.buildServer.vcs.impl.BuildChangesLoaderContext$BuildTypeCollectorResult.reThrowProblemException(BuildChangesLoaderContext.java:40)      at  jetbrains.buildServer.vcs.impl.BuildChangesLoaderContext$BuildTypeCollectorResult.access$1100(BuildChangesLoaderContext.java:35)      at jetbrains.buildServer.vcs.impl.BuildChangesLoaderContext.checkNoProblemsInMyBuildTypes(BuildChangesLoaderContext.java:43)      at jetbrains.buildServer.vcs.impl.BuildChangesLoaderContext.checkingForChangesFinished(BuildChangesLoaderContext.java:60)      at jetbrains.buildServer.vcs.impl.VcsChangesCollectorImpl$1.call(VcsChangesCollectorImpl.java:4)      at jetbrains.buildServer.vcs.impl.VcsChangesCollectorImpl$1.call(VcsChangesCollectorImpl.java)      ... 5 more Caused by: jetbrains.buildServer.vcs.VcsException: 'git fetch' command failed. stderr: java.lang.OutOfMemoryError: Java heap space      at org.eclipse.jgit.storage.pack.BinaryDelta.apply(BinaryDelta.java:163)      at org.eclipse.jgit.storage.pack.BinaryDelta.apply(BinaryDelta.java:118)      at org.eclipse.jgit.transport.PackParser.resolveDeltas(PackParser.java:561)      at org.eclipse.jgit.transport.PackParser.resolveDeltas(PackParser.java:544)      at org.eclipse.jgit.transport.PackParser.resolveDeltas(PackParser.java:507)      at org.eclipse.jgit.transport.PackParser.parse(PackParser.java:462)      at org.eclipse.jgit.storage.file.ObjectDirectoryPackParser.parse(ObjectDirectoryPackParser.java:178)      at org.eclipse.jgit.transport.PackParser.parse(PackParser.java:410)      at org.eclipse.jgit.transport.BasePackFetchConnection.receivePack(BasePackFetchConnection.java:649)      at org.eclipse.jgit.transport.BasePackFetchConnection.doFetch(BasePackFetchConnection.java:280)      at org.eclipse.jgit.transport.BasePackFetchConnection.fetch(BasePackFetchConnection.java:225)      at org.eclipse.jgit.transport.FetchProcess.fetchObjects(FetchProcess.java:214)      at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:149)      at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:111)      at org.eclipse.jgit.transport.Transport.fetch(Transport.java:903)      at jetbrains.buildServer.buildTriggers.vcs.git.Fetcher.fetch(Fetcher.java:81)      at jetbrains.buildServer.buildTriggers.vcs.git.Fetcher.main(Fetcher.java:47)      at jetbrains.buildServer.buildTriggers.vcs.git.CommandLineUtil.getCommandLineError(CommandLineUtil.java:41)      at jetbrains.buildServer.buildTriggers.vcs.git.FetchCommandImpl.fetchInSeparateProcess(FetchCommandImpl.java:129)      at jetbrains.buildServer.buildTriggers.vcs.git.FetchCommandImpl.fetch(FetchCommandImpl.java:68)      at jetbrains.buildServer.buildTriggers.vcs.git.GitVcsSupport.fetch(GitVcsSupport.java:1078)      at jetbrains.buildServer.buildTriggers.vcs.git.GitVcsSupport.fetchBranchData(GitVcsSupport.java:1060)      at jetbrains.buildServer.buildTriggers.vcs.git.GitVcsSupport.getCurrentVersion(GitVcsSupport.java:991)      at  jetbrains.buildServer.vcs.impl.VcsRootInstancesManagerImpl$SVcsRootInstance.getCurrentRevision(VcsRootInstancesManagerImpl.java:25)      at jetbrains.buildServer.vcs.impl.VcsManagerImpl.getRevisionsForAllRoots(VcsManagerImpl.java:181)      at jetbrains.buildServer.vcs.impl.VcsManagerImpl.loadChanges(VcsManagerImpl.java:709)      at jetbrains.buildServer.serverSide.impl.auth.SecuredVcsManager.loadChanges(SecuredVcsManager.java:64)      at jetbrains.buildServer.vcs.impl.VcsChangesCollectorImpl$2.call(VcsChangesCollectorImpl.java)      at jetbrains.buildServer.vcs.impl.VcsChangesCollectorImpl$2.call(VcsChangesCollectorImpl.java:1)      ... 5 more

[11:27:59]:
java.util.concurrent.ExecutionException:  jetbrains.buildServer.vcs.VcsException: Problem collecting changes for  'M2 Core :: ray_testing' : 'git fetch' command failed. stderr: java.lang.OutOfMemoryError: Java heap space      at org.eclipse.jgit.storage.pack.BinaryDelta.apply(BinaryDelta.java:163)      at org.eclipse.jgit.storage.pack.BinaryDelta.apply(BinaryDelta.java:118)      at org.eclipse.jgit.transport.PackParser.resolveDeltas(PackParser.java:561)      at org.eclipse.jgit.transport.PackParser.resolveDeltas(PackParser.java:544)      at org.eclipse.jgit.transport.PackParser.resolveDeltas(PackParser.java:507)      at org.eclipse.jgit.transport.PackParser.parse(PackParser.java:462)      at org.eclipse.jgit.storage.file.ObjectDirectoryPackParser.parse(ObjectDirectoryPackParser.java:178)      at org.eclipse.jgit.transport.PackParser.parse(PackParser.java:410)      at org.eclipse.jgit.transport.BasePackFetchConnection.receivePack(BasePackFetchConnection.java:649)      at org.eclipse.jgit.transport.BasePackFetchConnection.doFetch(BasePackFetchConnection.java:280)      at org.eclipse.jgit.transport.BasePackFetchConnection.fetch(BasePackFetchConnection.java:225)      at org.eclipse.jgit.transport.FetchProcess.fetchObjects(FetchProcess.java:214)      at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:149)      at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:111)      at org.eclipse.jgit.transport.Transport.fetch(Transport.java:903)      at jetbrains.buildServer.buildTriggers.vcs.git.Fetcher.fetch(Fetcher.java:81)      at jetbrains.buildServer.buildTriggers.vcs.git.Fetcher.main(Fetcher.java:47) java.util.concurrent.ExecutionException:  jetbrains.buildServer.vcs.VcsException: Problem collecting changes for  'M2 Core :: ray_testing' : 'git fetch' command failed. stderr: java.lang.OutOfMemoryError: Java heap space      at org.eclipse.jgit.storage.pack.BinaryDelta.apply(BinaryDelta.java:163)      at org.eclipse.jgit.storage.pack.BinaryDelta.apply(BinaryDelta.java:118)      at org.eclipse.jgit.transport.PackParser.resolveDeltas(PackParser.java:561)      at org.eclipse.jgit.transport.PackParser.resolveDeltas(PackParser.java:544)      at org.eclipse.jgit.transport.PackParser.resolveDeltas(PackParser.java:507)      at org.eclipse.jgit.transport.PackParser.parse(PackParser.java:462)      at org.eclipse.jgit.storage.file.ObjectDirectoryPackParser.parse(ObjectDirectoryPackParser.java:178)      at org.eclipse.jgit.transport.PackParser.parse(PackParser.java:410)      at org.eclipse.jgit.transport.BasePackFetchConnection.receivePack(BasePackFetchConnection.java:649)      at org.eclipse.jgit.transport.BasePackFetchConnection.doFetch(BasePackFetchConnection.java:280)      at org.eclipse.jgit.transport.BasePackFetchConnection.fetch(BasePackFetchConnection.java:225)      at org.eclipse.jgit.transport.FetchProcess.fetchObjects(FetchProcess.java:214)      at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:149)      at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:111)      at org.eclipse.jgit.transport.Transport.fetch(Transport.java:903)      at jetbrains.buildServer.buildTriggers.vcs.git.Fetcher.fetch(Fetcher.java:81)      at jetbrains.buildServer.buildTriggers.vcs.git.Fetcher.main(Fetcher.java:47)      at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)      at java.util.concurrent.FutureTask.get(FutureTask.java:83)      at jetbrains.buildServer.serverSide.impl.CancelableTaskHolder.waitForTaskToComplete(CancelableTaskHolder.java:1)      at  jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl.doCollectChanges(TopBuildDependencyGraphImpl.java:9)      at jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl.access$000(TopBuildDependencyGraphImpl.java:29)      at jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl$3.run(TopBuildDependencyGraphImpl.java:1)      at jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl$3.run(TopBuildDependencyGraphImpl.java)      at jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl$4.run(TopBuildDependencyGraphImpl.java:1)      at jetbrains.buildServer.serverSide.impl.auth.SecurityContextImpl.runAs(SecurityContextImpl.java:21)      at jetbrains.buildServer.serverSide.impl.auth.SecurityContextImpl.runAsSystem(SecurityContextImpl.java:1)      at  jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl.runAsSystem(TopBuildDependencyGraphImpl.java:66)      at  jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl.collectChangesForGraph(TopBuildDependencyGraphImpl.java:82)      at  jetbrains.buildServer.serverSide.impl.dependency.TopBuildDependencyGraphImpl.collectChangesForGraph(TopBuildDependencyGraphImpl.java:27)      at jetbrains.buildServer.serverSide.impl.BuildChangesCollector.collectChangesForBuild(BuildChangesCollector.java:21)      at jetbrains.buildServer.serverSide.impl.BuildStarter$1.run(BuildStarter.java:18)      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)      at java.util.concurrent.FutureTask.run(FutureTask.java:138)      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)      at java.lang.Thread.run(Thread.java:619) Caused by: jetbrains.buildServer.vcs.VcsException: Problem collecting  changes for 'M2 Core :: ray_testing' : 'git fetch' command failed. stderr: java.lang.OutOfMemoryError: Java heap space      at org.eclipse.jgit.storage.pack.BinaryDelta.apply(BinaryDelta.java:163)      at org.eclipse.jgit.storage.pack.BinaryDelta.apply(BinaryDelta.java:118)      at org.eclipse.jgit.transport.PackParser.resolveDeltas(PackParser.java:561)      at org.eclipse.jgit.transport.PackParser.resolveDeltas(PackParser.java:544)      at org.eclipse.jgit.transport.PackParser.resolveDeltas(PackParser.java:507)      at org.eclipse.jgit.transport.PackParser.parse(PackParser.java:462)      at org.eclipse.jgit.storage.file.ObjectDirectoryPackParser.parse(ObjectDirectoryPackParser.java:178)      at org.eclipse.jgit.transport.PackParser.parse(PackParser.java:410)      at org.eclipse.jgit.transport.BasePackFetchConnection.receivePack(BasePackFetchConnection.java:649)      at org.eclipse.jgit.transport.BasePackFetchConnection.doFetch(BasePackFetchConnection.java:280)      at org.eclipse.jgit.transport.BasePackFetchConnection.fetch(BasePackFetchConnection.java:225)      at org.eclipse.jgit.transport.FetchProcess.fetchObjects(FetchProcess.java:214)      at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:149)      at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:111)      at org.eclipse.jgit.transport.Transport.fetch(Transport.java:903)      at jetbrains.buildServer.buildTriggers.vcs.git.Fetcher.fetch(Fetcher.java:81)      at jetbrains.buildServer.buildTriggers.vcs.git.Fetcher.main(Fetcher.java:47)      at  jetbrains.buildServer.vcs.impl.BuildChangesLoaderContext$BuildTypeCollectorResult.reThrowProblemException(BuildChangesLoaderContext.java:40)      at  jetbrains.buildServer.vcs.impl.BuildChangesLoaderContext$BuildTypeCollectorResult.access$1100(BuildChangesLoaderContext.java:35)      at jetbrains.buildServer.vcs.impl.BuildChangesLoaderContext.checkNoProblemsInMyBuildTypes(BuildChangesLoaderContext.java:43)      at jetbrains.buildServer.vcs.impl.BuildChangesLoaderContext.checkingForChangesFinished(BuildChangesLoaderContext.java:60)      at jetbrains.buildServer.vcs.impl.VcsChangesCollectorImpl$1.call(VcsChangesCollectorImpl.java:4)      at jetbrains.buildServer.vcs.impl.VcsChangesCollectorImpl$1.call(VcsChangesCollectorImpl.java)      ... 5 more Caused by: jetbrains.buildServer.vcs.VcsException: 'git fetch' command failed. stderr: java.lang.OutOfMemoryError: Java heap space      at org.eclipse.jgit.storage.pack.BinaryDelta.apply(BinaryDelta.java:163)      at org.eclipse.jgit.storage.pack.BinaryDelta.apply(BinaryDelta.java:118)      at org.eclipse.jgit.transport.PackParser.resolveDeltas(PackParser.java:561)      at org.eclipse.jgit.transport.PackParser.resolveDeltas(PackParser.java:544)      at org.eclipse.jgit.transport.PackParser.resolveDeltas(PackParser.java:507)      at org.eclipse.jgit.transport.PackParser.parse(PackParser.java:462)      at org.eclipse.jgit.storage.file.ObjectDirectoryPackParser.parse(ObjectDirectoryPackParser.java:178)      at org.eclipse.jgit.transport.PackParser.parse(PackParser.java:410)      at org.eclipse.jgit.transport.BasePackFetchConnection.receivePack(BasePackFetchConnection.java:649)      at org.eclipse.jgit.transport.BasePackFetchConnection.doFetch(BasePackFetchConnection.java:280)      at org.eclipse.jgit.transport.BasePackFetchConnection.fetch(BasePackFetchConnection.java:225)      at org.eclipse.jgit.transport.FetchProcess.fetchObjects(FetchProcess.java:214)      at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:149)      at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:111)      at org.eclipse.jgit.transport.Transport.fetch(Transport.java:903)      at jetbrains.buildServer.buildTriggers.vcs.git.Fetcher.fetch(Fetcher.java:81)      at jetbrains.buildServer.buildTriggers.vcs.git.Fetcher.main(Fetcher.java:47)      at jetbrains.buildServer.buildTriggers.vcs.git.CommandLineUtil.getCommandLineError(CommandLineUtil.java:41)      at jetbrains.buildServer.buildTriggers.vcs.git.FetchCommandImpl.fetchInSeparateProcess(FetchCommandImpl.java:129)      at jetbrains.buildServer.buildTriggers.vcs.git.FetchCommandImpl.fetch(FetchCommandImpl.java:68)      at jetbrains.buildServer.buildTriggers.vcs.git.GitVcsSupport.fetch(GitVcsSupport.java:1078)      at jetbrains.buildServer.buildTriggers.vcs.git.GitVcsSupport.fetchBranchData(GitVcsSupport.java:1060)      at jetbrains.buildServer.buildTriggers.vcs.git.GitVcsSupport.getCurrentVersion(GitVcsSupport.java:991)      at  jetbrains.buildServer.vcs.impl.VcsRootInstancesManagerImpl$SVcsRootInstance.getCurrentRevision(VcsRootInstancesManagerImpl.java:25)      at jetbrains.buildServer.vcs.impl.VcsManagerImpl.getRevisionsForAllRoots(VcsManagerImpl.java:181)      at jetbrains.buildServer.vcs.impl.VcsManagerImpl.loadChanges(VcsManagerImpl.java:709)      at jetbrains.buildServer.serverSide.impl.auth.SecuredVcsManager.loadChanges(SecuredVcsManager.java:64)      at jetbrains.buildServer.vcs.impl.VcsChangesCollectorImpl$2.call(VcsChangesCollectorImpl.java)      at jetbrains.buildServer.vcs.impl.VcsChangesCollectorImpl$2.call(VcsChangesCollectorImpl.java:1)

0
Comment actions Permalink

Hi Raymond,

it is really strange that you have OutOfMemoryError with your settings. Stacktrace shows that error occurs during parsing pack file, received from remote repository. Since you cloned repository manually, this pack file should be small and should not cause any problems. Did make a bare clone of your repository?

0
Comment actions Permalink

I had not -- I made a regular clone.  Trying it bare now.

0
Comment actions Permalink

It's been 'checking changes' now for nearly an hour.  I'll let it run a while and see what it does.  With this repo it takes a long time even with git at the CLI -- though not nearly this long, 5 minutes or so -- so we'll see how it goes.  At least the memory issue seems to be resolved.  I'll let you know if there are more issues, but in the mean time, thank you for your help!

0
Comment actions Permalink

Hi Raymond,

did 'checking for changes' process finished? I can think of one reason of why it is so slow: submodules. Do you use them?

0
Comment actions Permalink

No submodules.  The bare clone did the trick, altough it ran about 4 hours before I killed it, started it again and it came through finally about a half hour later.

I then ran into more memory issues running another project that depended on the first (Both Rails 3 on jRuby).  I was getting a GC Memory error when running rake:integration tests, using Selenium and a headless XTerm/Firefox setup.  When I finally gave up the rake step and instead used a CLI step, calling 'ruby -XmxXXX -S rake', bumping up the available memory to the Rails app I got it to run finally.  jRuby 1.6.2/RVM setup...

Thank you for checking back.  This was probably the toughest install of anything that I've done in 20 years!  Next:  MSDev on a Blackberry =)  I am looking forward to using TC -- it seems well thought out.

0
Comment actions Permalink

Raymond,
I think checking for changes shouldn't be so long. If you will have this problem once again, please take few thread dumps (you can do it on the Administration > Server Configuration > Diagnostics) and provide a teamcity-vcs.log.
Also I fixed a few more bugs recently, you can take the last build of git-plugin with fixes from http://teamcity.jetbrains.com/viewType.html?buildTypeId=bt245&tab=buildTypeStatusDiv.

0

Please sign in to leave a comment.