1. TeamCity Support | JetBrains
  2. Community
  3. TeamCity General Topics

Server can't start due to java.lang.OutOfMemoryError: Java heap space

Answered
Avatar
Murph269
Created

I've seen similar issues reported, but this one is preventing the server from starting instead of just preventing builds from running. This is a server running on Windows. I tried changing TEAMCITY_SERVER_MEM_OPTS to -Xmx4g but it made no difference. When I attempt to start my server, I get a page that reads as follows:

Unexpected Error

This was not supposed to happen. Please provide the error details to your TeamCity server maintainer.
If you maintain this TeamCity installation and the request seems legit please report this error to JetBrains.

Error: jetbrains.buildServer.serverSide.db.UnexpectedDBException: Unexpected exception: SQL error when doing 'Executing with parameters: [bt17] caused SQL error: ' while performing SQL query: select bs.build_id as build_id, bs.modification_id as mod_id, bs.chain_modification_id as chain_mod_id, bs.branch_name as branch, bs.id as promotion_id, h.is_canceled as canceled, bs.is_personal as personal, bs.is_changes_detached as changes_detached, bs.build_type_id as build_type_id, bs.remove_from_queue_time as remove_from_queue_time, h.status as status, revs.vcs_root_id as rev_root_id, revs.vcs_revision as rev_revision, revs.modification_id as rev_mod_id, revs.checkout_mode as rev_checkout_mode from build_state bs left join build_revisions revs on revs.build_state_id = bs.id left join history h on h.build_id = bs.build_id where bs.build_type_id = ? and bs.is_deleted = 0 and bs.build_id is not null order by bs.build_id : java.sql.SQLException: java.lang.OutOfMemoryError: Java heap space. Caused by: org.hsqldb.HsqlException: java.lang.OutOfMemoryError: Java heap space

Server time: 2020-06-10 09:27:21
TeamCity: 2019.2.3 (build 72031)
Servlet container: Apache Tomcat/8.5.47

Trace: jetbrains.buildServer.serverSide.db.UnexpectedDBException: Unexpected exception: SQL error when doing 'Executing  with parameters: [bt17] caused SQL error: ' while performing SQL query: select bs.build_id as build_id,  bs.modification_id as mod_id,  bs.chain_modification_id as chain_mod_id,  bs.branch_name as branch,  bs.id as promotion_id,  h.is_canceled as canceled,  bs.is_personal as personal,  bs.is_changes_detached as changes_detached,  bs.build_type_id as build_type_id,  bs.remove_from_queue_time as remove_from_queue_time,  h.status as status,  revs.vcs_root_id as rev_root_id,  revs.vcs_revision as rev_revision,  revs.modification_id as rev_mod_id,  revs.checkout_mode as rev_checkout_mode from build_state bs left join build_revisions revs on revs.build_state_id = bs.id left join history h on h.build_id = bs.build_id where bs.build_type_id = ? and bs.is_deleted = 0 and bs.build_id is not null order by bs.build_id : java.sql.SQLException: java.lang.OutOfMemoryError: Java heap space. Caused by: org.hsqldb.HsqlException: java.lang.OutOfMemoryError: Java heap space
	at org.hsqldb.jdbc.JDBCUtil.sqlException(JDBCUtil.java:418)
	at org.hsqldb.jdbc.JDBCUtil.sqlException(JDBCUtil.java:247)
	at org.hsqldb.jdbc.JDBCPreparedStatement.fetchResult(JDBCPreparedStatement.java:4655)
	at org.hsqldb.jdbc.JDBCPreparedStatement.execute(JDBCPreparedStatement.java:1098)
	at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:191)
	at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:191)
	at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:191)
	at jetbrains.buildServer.serverSide.db.queries.GenericQuery$8.run(GenericQuery.java:356)
	at jetbrains.buildServer.serverSide.db.queries.GenericQuery$8.run(GenericQuery.java:353)
	at jetbrains.buildServer.serverSide.db.DBInternalUtils.rawOperation(DBInternalUtils.java:70)
	at jetbrains.buildServer.serverSide.db.queries.GenericQuery.doExecuteQuery(GenericQuery.java:353)
	at jetbrains.buildServer.serverSide.db.queries.GenericQuery.access$500(GenericQuery.java:27)
	at jetbrains.buildServer.serverSide.db.queries.GenericQuery$7.action(GenericQuery.java:339)
	at jetbrains.buildServer.serverSide.db.queries.GenericQuery.underPreparedStatement(GenericQuery.java:409)
	at jetbrains.buildServer.serverSide.db.queries.GenericQuery.doExecuteQuery(GenericQuery.java:336)
	at jetbrains.buildServer.serverSide.db.queries.GenericQuery.execute(GenericQuery.java:78)
	at jetbrains.buildServer.serverSide.impl.BuildTypeOrderedBuildsImpl.lambda$loadBuilds$4(BuildTypeOrderedBuildsImpl.java:108)
	at jetbrains.buildServer.serverSide.db.InternalDBUtil.runAndRetry(InternalDBUtil.java:25)
	at jetbrains.buildServer.serverSide.db.TeamCityDatabaseManager.runAndRetry(TeamCityDatabaseManager.java:1142)
	at jetbrains.buildServer.serverSide.db.DBFacade.runAndRetry(DBFacade.java:292)
	at jetbrains.buildServer.serverSide.impl.BuildTypeOrderedBuildsImpl.loadBuilds(BuildTypeOrderedBuildsImpl.java:143)
	at jetbrains.buildServer.serverSide.impl.BuildTypeOrderedBuildsImpl.getBuilds(BuildTypeOrderedBuildsImpl.java:71)
	at jetbrains.buildServer.serverSide.impl.BuildTypeOrderedBuildsImpl.consumeBuilds(BuildTypeOrderedBuildsImpl.java:35)
	at jetbrains.buildServer.serverSide.impl.BuildTypeBranches.getRecentlyUsedBuildBranches(BuildTypeBranches.java:151)
	at jetbrains.buildServer.serverSide.impl.BuildTypeBranches.access$200(BuildTypeBranches.java:262)
	at jetbrains.buildServer.serverSide.impl.BuildTypeBranches$ActiveBuildBranches.readFromDb(BuildTypeBranches.java:40)
	at jetbrains.buildServer.serverSide.impl.BuildTypeBranches$ActiveBuildBranches.getValue(BuildTypeBranches.java:38)
	at jetbrains.buildServer.serverSide.impl.BuildTypeBranches.getBranchInfo(BuildTypeBranches.java:212)
	at jetbrains.buildServer.serverSide.impl.BuildTypeBranches.getBranches(BuildTypeBranches.java:46)
	at jetbrains.buildServer.serverSide.impl.BuildTypeImpl.getBranches(BuildTypeImpl.java:508)
	at jetbrains.buildServer.controllers.project.ProjectWebUtil.lambda$projectBranchesAnyMatch$2(ProjectWebUtil.java:27)
	at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:269)
	at java.util.ArrayList$ArrayListSpliterator.tryAdvance(ArrayList.java:1359)
	at java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126)
	at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:499)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:486)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
	at java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:230)
	at java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:196)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.anyMatch(ReferencePipeline.java:516)
	at jetbrains.buildServer.controllers.project.ProjectWebUtil.projectBranchesAnyMatch(ProjectWebUtil.java:10)
	at jetbrains.buildServer.controllers.project.ProjectWebUtil.getBranchBeansForProjects(ProjectWebUtil.java:43)
	at jetbrains.buildServer.controllers.overview.OverviewBean.<init>(OverviewBean.java:85)
	at jetbrains.buildServer.controllers.overview.OverviewController.getOverviewBean(OverviewController.java:28)
	at jetbrains.buildServer.controllers.overview.OverviewController.doHandle(OverviewController.java:47)
	at jetbrains.buildServer.controllers.BaseController.handleRequestInternal(BaseController.java:114)
	at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:174)
	at jetbrains.buildServer.controllers.BaseController.handleRequest(BaseController.java:93)
	at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:50)
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
	at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:634)
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
	at jetbrains.buildServer.maintenance.TeamCityDispatcherServlet.processedByMainServlet(TeamCityDispatcherServlet.java:1)
	at jetbrains.buildServer.maintenance.TeamCityDispatcherServlet.service(TeamCityDispatcherServlet.java:14)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at jetbrains.buildServer.web.jsp.JspPrecompilerFilter.doFilter(JspPrecompilerFilter.java:104)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at jetbrains.buildServer.web.DisableSessionIdFromUrlFilter.doFilter(DisableSessionIdFromUrlFilter.java:5)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at jetbrains.buildServer.web.UserIdProviderFilter.doFilter(UserIdProviderFilter.java:12)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at jetbrains.buildServer.web.NodeInfoHeaderFilter.doFilter(NodeInfoHeaderFilter.java:9)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:107)
	at jetbrains.buildServer.diagnostic.web.DiagnosticFilter.doFilter(DiagnosticFilter.java:54)
	at org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:112)
	at jetbrains.buildServer.web.DependencyParametersCalculationContextFilter.doFilter(DependencyParametersCalculationContextFilter.java:2)
	at org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:112)
	at jetbrains.buildServer.diagnostic.web.HttpRequestsDurationMetricsReporter.doFilter(HttpRequestsDurationMetricsReporter.java:9)
	at org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:112)
	at jetbrains.buildServer.web.HttpSecurityHeadersFilter.doFilter(HttpSecurityHeadersFilter.java:62)
	at org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:112)
	at org.springframework.web.filter.CompositeFilter.doFilter(CompositeFilter.java:73)
	at jetbrains.buildServer.web.DelegatingFilter.doFilter(DelegatingFilter.java:61)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at jetbrains.buildServer.web.ResponseFragmentFilter.doFilter(ResponseFragmentFilter.java:43)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:528)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:798)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:810)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1500)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.sql.SQLException: java.lang.OutOfMemoryError: Java heap space
	... 108 more
Caused by: org.hsqldb.HsqlException: java.lang.OutOfMemoryError: Java heap space
	at org.hsqldb.error.Error.error(Error.java:113)
	at org.hsqldb.persist.DataFileCache.getFromFile(DataFileCache.java:1078)
	at org.hsqldb.persist.DataFileCache.get(DataFileCache.java:1042)
	at org.hsqldb.persist.RowStoreAVLDisk.get(RowStoreAVLDisk.java:118)
	at org.hsqldb.index.NodeAVLDisk.findNode(NodeAVLDisk.java:186)
	at org.hsqldb.index.NodeAVLDisk.getRight(NodeAVLDisk.java:250)
	at org.hsqldb.index.IndexAVL.findNode(IndexAVL.java:1742)
	at org.hsqldb.index.IndexAVL.findFirstRow(IndexAVL.java:1171)
	at org.hsqldb.RangeVariable$RangeIteratorMain.getFirstRow(RangeVariable.java:1356)
	at org.hsqldb.RangeVariable$RangeIteratorMain.initialiseIterator(RangeVariable.java:1265)
	at org.hsqldb.RangeVariable$RangeIteratorMain.next(RangeVariable.java:1203)
	at org.hsqldb.QuerySpecification.buildResult(QuerySpecification.java:1533)
	at org.hsqldb.QuerySpecification.getSingleResult(QuerySpecification.java:1439)
	at org.hsqldb.QuerySpecification.getResult(QuerySpecification.java:1429)
	at org.hsqldb.StatementQuery.getResult(StatementQuery.java:66)
	at org.hsqldb.StatementDMQL.execute(StatementDMQL.java:195)
	at org.hsqldb.Session.executeCompiledStatement(Session.java:1378)
	at org.hsqldb.Session.execute(Session.java:991)
	at org.hsqldb.jdbc.JDBCPreparedStatement.fetchResult(JDBCPreparedStatement.java:4647)
	... 105 more
Caused by: java.lang.OutOfMemoryError: Java heap space
1 comment
Avatar
Denis Lapuente
Created
Comment actions Permalink

Hi,

 

thanks for your report. The "Java heap space" error typically indicates that the server does not have enough memory to run. If you added the TEAMCITY_SERVER_MEM_OPTS, there are two points to keep in mind:

-This environment variable has to be picked up by the process starting up. If you run the server as a service, you might need to adapt the user that runs it, restart the server, etc. During the startup process, the server logs the "JVM parameters". You can find this line and see if the -Xmx is being applied or not. You can also test by stopping the service (if applies), opening a command line, exporting the variable there and from the same shell spawning the server, which should then inherit it.

-Any amount above 2Gb in theory, around 1 - 1.2Gb in practice, will only work if you are using a 64 bit Java, since 32 bit cannot manage so much memory. If your installation is using 32 bit java it will not work, fail with the same issue, but require changing to 64 to work. We have instructions for it here: https://www.jetbrains.com/help/teamcity/installing-and-configuring-the-teamcity-server.html#InstallingandConfiguringtheTeamCityServer-Using64bitJavatoRunTeamCityServer

 

Furthermore, the HSQLDB system is not recommended for production servers. If you are still using the integrated database, please consider migrating it to an external database: https://www.jetbrains.com/help/teamcity/migrating-to-an-external-database.html

0

Please sign in to leave a comment.