Build Agent wont start - Build queued

Avatar
Permanently deleted user
Created

Hi

I have configured a build agent to run a set of builds.  It is configured properly with no errors or conflicts and it shows the builds as being queued but it will not start them.

I started having this issue this weekend with another set of mature builds that have been running for some time.  I had to recycle the team city web server to get them going?

Any ideas as to what is happening?

Votes

0

Share

17 comments
Avatar
Pavel Sher
Created

In the build queue if you move your mouse over the Time to start value a popup should appear explaining why the build is not starting.

0
Avatar
Permanently deleted user
Created

It just says delayed.   It shows in the queue as being ready.  It shows how long the build will take in the case of a mature build but it will not start.   My solution is to restart team city server.

0
Avatar
Pavel Sher
Created

But what is shown in the tooltip? Usually for delayed builds there are details why they are delayed. Can you provide a screenshot?

0
Avatar
Stian Fagereng
Created

I got the same behaviour with version 7.0.3.

Screenshot of my queue.
Untitled.png

My agents just sitting idle
Agents.png



Hangs and Thread Dumps


2012-06-14 12:28:47
Full thread dump Java HotSpot(TM) Server VM (20.6-b01 mixed mode):

"Normal executor 5" group="main" prio=5 tid=87 nid=87 waiting 
   java.lang.Thread.State: WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
    at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"VCS executor 4" group="main" prio=5 tid=88 nid=88 waiting 
   java.lang.Thread.State: WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
    at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"http-bio-8088-exec-28" daemon group="main" prio=5 tid=4 569 nid=4 569 runnable 
   java.lang.Thread.State: RUNNABLE
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(Unknown Source)
    at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:516)
    at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:501)
    at org.apache.coyote.http11.Http11Processor.setRequestLineReadTimeout(Http11Processor.java:172)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:912)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:539)
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:300)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"NuGet Packages Version Checker 4" daemon group="main" prio=5 tid=142 nid=142 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown Source)
    at java.util.concurrent.DelayQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"http-bio-8088-exec-1" daemon group="main" prio=5 tid=19 nid=19 runnable 
   java.lang.Thread.State: RUNNABLE
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(Unknown Source)
    at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:516)
    at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:501)
    at org.apache.coyote.http11.Http11Processor.setRequestLineReadTimeout(Http11Processor.java:172)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:912)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:539)
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:300)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Normal executor 6" group="main" prio=5 tid=93 nid=93 waiting 
   java.lang.Thread.State: WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
    at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Normal executor 3" group="main" prio=5 tid=80 nid=80 waiting 
   java.lang.Thread.State: WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
    at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"http-bio-8088-exec-11" daemon group="main" prio=5 tid=94 nid=94 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown Source)
    at java.util.concurrent.LinkedBlockingQueue.poll(Unknown Source)
    at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
    at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"main" group="main" prio=5 tid=1 nid=1 runnable 
   java.lang.Thread.State: RUNNABLE
    at java.net.PlainSocketImpl.socketAccept(Native Method)
    at java.net.PlainSocketImpl.accept(Unknown Source)
    at java.net.ServerSocket.implAccept(Unknown Source)
    at java.net.ServerSocket.accept(Unknown Source)
    at org.apache.catalina.core.StandardServer.await(StandardServer.java:447)
    at org.apache.catalina.startup.Catalina.await(Catalina.java:706)
    at org.apache.catalina.startup.Catalina.start(Catalina.java:652)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:322)
    at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:450)

"Alarm (NotificationThread-HangedBuilds)" daemon group="main" prio=5 tid=38 nid=38 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown Source)
    at java.util.concurrent.DelayQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"VCS executor 2" group="main" prio=5 tid=81 nid=81 waiting 
   java.lang.Thread.State: WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
    at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"VCS executor 5" group="main" prio=5 tid=120 nid=120 waiting 
   java.lang.Thread.State: WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
    at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"http-bio-8088-exec-30" daemon group="main" prio=5 tid=4 731 nid=4 731 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown Source)
    at java.util.concurrent.LinkedBlockingQueue.poll(Unknown Source)
    at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
    at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Alarm (VersionChecker)" daemon group="main" prio=5 tid=77 nid=77 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown Source)
    at java.util.concurrent.DelayQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Attach Listener" daemon group="system" prio=5 tid=5 nid=5 runnable 
   java.lang.Thread.State: RUNNABLE

"http-bio-8088-exec-3" daemon group="main" prio=5 tid=21 nid=21 runnable 
   java.lang.Thread.State: RUNNABLE
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(Unknown Source)
    at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:516)
    at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:501)
    at org.apache.coyote.http11.Http11Processor.setRequestLineReadTimeout(Http11Processor.java:172)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:912)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:539)
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:300)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"http-bio-8088-exec-23" daemon group="main" prio=5 tid=333 nid=333 runnable 
   java.lang.Thread.State: RUNNABLE
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(Unknown Source)
    at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:516)
    at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:501)
    at org.apache.coyote.http11.Http11Processor.setRequestLineReadTimeout(Http11Processor.java:172)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:912)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:539)
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:300)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Flush Queue" daemon group="main" prio=5 tid=50 nid=50 runnable 
   java.lang.Thread.State: RUNNABLE
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(Unknown Source)
    at java.io.DataInputStream.readFully(Unknown Source)
    at java.io.DataInputStream.readFully(Unknown Source)
    at net.sourceforge.jtds.jdbc.SharedSocket.readPacket(SharedSocket.java:846)
    at net.sourceforge.jtds.jdbc.SharedSocket.getNetPacket(SharedSocket.java:727)
    at net.sourceforge.jtds.jdbc.ResponseStream.getPacket(ResponseStream.java:466)
    at net.sourceforge.jtds.jdbc.ResponseStream.read(ResponseStream.java:103)
    at net.sourceforge.jtds.jdbc.ResponseStream.peek(ResponseStream.java:88)
    at net.sourceforge.jtds.jdbc.TdsCore.wait(TdsCore.java:3932)
    at net.sourceforge.jtds.jdbc.TdsCore.executeSQL(TdsCore.java:1046)
    at net.sourceforge.jtds.jdbc.MSCursorResultSet.cursorCreate(MSCursorResultSet.java:536)
    at net.sourceforge.jtds.jdbc.MSCursorResultSet.<init>(MSCursorResultSet.java:154)
    at net.sourceforge.jtds.jdbc.JtdsStatement.executeSQLQuery(JtdsStatement.java:424)
    at net.sourceforge.jtds.jdbc.JtdsStatement.executeQuery(JtdsStatement.java:1304)
    at org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
    at org.apache.commons.dbcp.PoolableConnectionFactory.validateConnection(PoolableConnectionFactory.java:658)
    at org.apache.commons.dbcp.PoolableConnectionFactory.validateObject(PoolableConnectionFactory.java:635)
    at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1198)
    at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:106)
    at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)
    at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111)
    at org.springframework.jdbc.datasource.TransactionAwareDataSourceProxy$TransactionAwareInvocationHandler.invoke(TransactionAwareDataSourceProxy.java:224)
    at $Proxy17.getMetaData(Unknown Source)
    at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.springframework.jdbc.core.JdbcTemplate$CloseSuppressingInvocationHandler.invoke(JdbcTemplate.java:1345)
    at $Proxy17.getMetaData(Unknown Source)
    at jetbrains.buildServer.serverSide.db.ConnectionUtil.detectDatabaseType(ConnectionUtil.java:18)
    at jetbrains.buildServer.serverSide.db.DBFunctions.create(DBFunctions.java:215)
    at jetbrains.buildServer.serverSide.db.DBFunctions.withDB(DBFunctions.java:2267)
    at jetbrains.buildServer.serverSide.db.queries.GenericQuery.underPreparedStatement(GenericQuery.java:469)
    at jetbrains.buildServer.serverSide.db.queries.GenericQuery.doExecuteQuery(GenericQuery.java:401)
    at jetbrains.buildServer.serverSide.db.queries.GenericQuery.access$100(GenericQuery.java:24)
    at jetbrains.buildServer.serverSide.db.queries.GenericQuery$1.run(GenericQuery.java:132)
    at jetbrains.buildServer.serverSide.db.DBFacade$6.doInConnection(DBFacade.java:407)
    at jetbrains.buildServer.serverSide.db.DBFacade$7.doInConnection(DBFacade.java:431)
    at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:341)
    at jetbrains.buildServer.serverSide.db.DBFacade._runSql(DBFacade.java:427)
    at jetbrains.buildServer.serverSide.db.DBFacade.runSql(DBFacade.java:405)
    at jetbrains.buildServer.serverSide.db.queries.GenericQuery.execute(GenericQuery.java:135)
    at jetbrains.buildServer.a.f.k(f.java:46)
    at jetbrains.buildServer.a.m.isMaxNumberOfAuthorizedAgentsExceeded(m.java:8)
    at jetbrains.buildServer.serverSide.impl.BuildServerImpl.flushQueue(BuildServerImpl.java:8)
    at jetbrains.buildServer.serverSide.impl.BuildServerRunner$2.doSomething(BuildServerRunner.java:1)
    at jetbrains.buildServer.serverSide.impl.BuildServerRunner$BuildServerWorker.run(BuildServerRunner.java:28)
    at java.lang.Thread.run(Unknown Source)

"Memory Usage Watcher" daemon group="main" prio=5 tid=76 nid=76 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at java.lang.Thread.sleep(Native Method)
    at jetbrains.buildServer.web.MemoryUsageMonitor$1.run(MemoryUsageMonitor.java:15)
    at java.lang.Thread.run(Unknown Source)

"Test failure rate persisting thread" daemon group="main" prio=5 tid=40 nid=40 waiting 
   java.lang.Thread.State: WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
    at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)
    at jetbrains.buildServer.serverSide.stat.TestFailureRateCollector.executeCommands(TestFailureRateCollector.java:112)
    at jetbrains.buildServer.serverSide.stat.TestFailureRateCollector.processPersistingQueue(TestFailureRateCollector.java:144)
    at jetbrains.buildServer.serverSide.stat.TestFailureRateCollector.access$400(TestFailureRateCollector.java:127)
    at jetbrains.buildServer.serverSide.stat.TestFailureRateCollector$4.run(TestFailureRateCollector.java)
    at java.lang.Thread.run(Unknown Source)

"ObsoleteBuildsKiller 1" group="main" prio=5 tid=53 nid=53 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown Source)
    at java.util.concurrent.DelayQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Usage Statistics Collector" group="main" prio=5 tid=46 nid=46 waiting 
   java.lang.Thread.State: WAITING
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:485)
    at jetbrains.buildServer.usageStatistics.impl.UsageStatisticsCollectorImpl.waitForEvent(UsageStatisticsCollectorImpl.java:199)
    at jetbrains.buildServer.usageStatistics.impl.UsageStatisticsCollectorImpl.run(UsageStatisticsCollectorImpl.java:150)
    at java.lang.Thread.run(Unknown Source)

"Low prio executor 1" daemon group="main" prio=4 tid=95 nid=95 waiting 
   java.lang.Thread.State: WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
    at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Low prio executor 5" daemon group="main" prio=4 tid=165 nid=165 waiting 
   java.lang.Thread.State: WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
    at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Normal executor 9" group="main" prio=5 tid=104 nid=104 waiting 
   java.lang.Thread.State: WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
    at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Store buildsToReindex Spool Thread" daemon group="main" prio=5 tid=78 nid=78 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at java.lang.Thread.sleep(Native Method)
    at net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:620)
    at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:66)
    at net.sf.ehcache.store.DiskStore$SpoolAndExpiryThread.run(DiskStore.java:1074)

"NuGet Packages Version Checker 3" daemon group="main" prio=5 tid=134 nid=134 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown Source)
    at java.util.concurrent.DelayQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Low prio executor 4" daemon group="main" prio=4 tid=164 nid=164 waiting 
   java.lang.Thread.State: WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
    at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"http-bio-8088-exec-26" daemon group="main" prio=5 tid=1 385 nid=1 385 runnable 
   java.lang.Thread.State: RUNNABLE
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(Unknown Source)
    at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:516)
    at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:501)
    at org.apache.coyote.http11.Http11Processor.setRequestLineReadTimeout(Http11Processor.java:172)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:912)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:539)
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:300)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"GC Daemon" daemon group="system" prio=2 tid=11 nid=11 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at java.lang.Object.wait(Native Method)
    at sun.misc.GC$Daemon.run(Unknown Source)

"Normal executor 8" group="main" prio=5 tid=102 nid=102 waiting 
   java.lang.Thread.State: WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
    at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Alarm (GetRunningBuildsAction Alarm)" daemon group="main" prio=5 tid=79 nid=79 waiting 
   java.lang.Thread.State: WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
    at java.util.concurrent.DelayQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Normal executor 10" group="main" prio=5 tid=114 nid=114 waiting 
   java.lang.Thread.State: WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
    at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"pool-11-thread-13" group="main" prio=5 tid=3 712 nid=3 712 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
    at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown Source)
    at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source)
    at java.util.concurrent.SynchronousQueue.poll(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"VCS executor 1" group="main" prio=5 tid=54 nid=54 waiting 
   java.lang.Thread.State: WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
    at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Normal executor 1" group="main" prio=5 tid=48 nid=48 waiting 
   java.lang.Thread.State: WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
    at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"NuGet Packages Version Checker 2" daemon group="main" prio=5 tid=127 nid=127 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown Source)
    at java.util.concurrent.DelayQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"http-bio-8088-exec-22" daemon group="main" prio=5 tid=332 nid=332 runnable 
   java.lang.Thread.State: RUNNABLE
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(Unknown Source)
    at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:516)
    at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:501)
    at org.apache.coyote.http11.Http11Processor.setRequestLineReadTimeout(Http11Processor.java:172)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:912)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:539)
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:300)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Store tables Spool Thread" daemon group="main" prio=5 tid=42 nid=42 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at java.lang.Thread.sleep(Native Method)
    at net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:620)
    at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:66)
    at net.sf.ehcache.store.DiskStore$SpoolAndExpiryThread.run(DiskStore.java:1074)

"Store Subversion Externals Spool Thread" daemon group="main" prio=5 tid=43 nid=43 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at java.lang.Thread.sleep(Native Method)
    at net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:620)
    at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:66)
    at net.sf.ehcache.store.DiskStore$SpoolAndExpiryThread.run(DiskStore.java:1074)

"Reference Handler" daemon group="system" prio=10 tid=2 nid=2 waiting 
   java.lang.Thread.State: WAITING
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:485)
    at java.lang.ref.Reference$ReferenceHandler.run(Unknown Source)

"Normal executor 4" group="main" prio=5 tid=85 nid=85 waiting 
   java.lang.Thread.State: WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
    at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Alarm (Disk space watcher)" daemon group="main" prio=5 tid=41 nid=41 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown Source)
    at java.util.concurrent.DelayQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Scheduled executor 3" group="main" prio=5 tid=36 nid=36 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown Source)
    at java.util.concurrent.DelayQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Build Triggers Monitor" daemon group="main" prio=5 tid=52 nid=52 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at java.lang.Object.wait(Native Method)
    at jetbrains.buildServer.serverSide.impl.BuildServerRunner$BuildServerWorker.run(BuildServerRunner.java:35)
    at java.lang.Thread.run(Unknown Source)

"http-bio-8088-exec-27" daemon group="main" prio=5 tid=4 568 nid=4 568 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown Source)
    at java.util.concurrent.LinkedBlockingQueue.poll(Unknown Source)
    at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
    at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Signal Dispatcher" daemon group="system" prio=9 tid=4 nid=4 runnable 
   java.lang.Thread.State: RUNNABLE

"ContainerBackgroundProcessor[StandardEngine[Catalina]]" daemon group="main" prio=5 tid=16 nid=16 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at java.lang.Thread.sleep(Native Method)
    at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1474)
    at java.lang.Thread.run(Unknown Source)

"Scheduled executor 4" group="main" prio=5 tid=37 nid=37 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown Source)
    at java.util.concurrent.DelayQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Store issue-errors Spool Thread" daemon group="main" prio=5 tid=45 nid=45 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at java.lang.Thread.sleep(Native Method)
    at net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:620)
    at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:66)
    at net.sf.ehcache.store.DiskStore$SpoolAndExpiryThread.run(DiskStore.java:1074)

"http-bio-8088-AsyncTimeout" daemon group="main" prio=5 tid=18 nid=18 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at java.lang.Thread.sleep(Native Method)
    at org.apache.tomcat.util.net.JIoEndpoint$AsyncTimeout.run(JIoEndpoint.java:136)
    at java.lang.Thread.run(Unknown Source)

"Normal executor 7" group="main" prio=5 tid=97 nid=97 waiting 
   java.lang.Thread.State: WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
    at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"SearchService-BackgroundIndexing" daemon group="main" prio=1 tid=75 nid=75 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at java.lang.Thread.sleep(Native Method)
    at jetbrains.buildServer.serverSide.search.BackgroundIndexer.run(BackgroundIndexer.java:33)
    at java.lang.Thread.run(Unknown Source)

"VCS executor 3" group="main" prio=5 tid=86 nid=86 waiting 
   java.lang.Thread.State: WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
    at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Thread-2" daemon group="main" prio=5 tid=14 nid=14 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at java.lang.Thread.sleep(Native Method)
    at org.apache.log4j.helpers.FileWatchdog.run(FileWatchdog.java:103)

"CommittedBuildsUpdater 1" group="main" prio=5 tid=92 nid=92 waiting 
   java.lang.Thread.State: WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
    at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Store provider-nuget Spool Thread" daemon group="main" prio=5 tid=168 nid=168 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at java.lang.Thread.sleep(Native Method)
    at net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:620)
    at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:66)
    at net.sf.ehcache.store.DiskStore$SpoolAndExpiryThread.run(DiskStore.java:1074)

"Vcs Monitor" daemon group="main" prio=5 tid=51 nid=51 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at java.lang.Object.wait(Native Method)
    at jetbrains.buildServer.serverSide.impl.BuildServerRunner$BuildServerWorker.run(BuildServerRunner.java:35)
    at java.lang.Thread.run(Unknown Source)

"http-bio-8088-exec-14" daemon group="main" prio=5 tid=105 nid=105 runnable 
   java.lang.Thread.State: RUNNABLE
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(Unknown Source)
    at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:516)
    at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:501)
    at org.apache.coyote.http11.Http11Processor.setRequestLineReadTimeout(Http11Processor.java:172)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:912)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:539)
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:300)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"http-bio-8088-exec-4" daemon group="main" prio=5 tid=22 nid=22 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown Source)
    at java.util.concurrent.LinkedBlockingQueue.poll(Unknown Source)
    at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
    at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Low prio executor 2" daemon group="main" prio=4 tid=103 nid=103 waiting 
   java.lang.Thread.State: WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
    at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"http-bio-8088-exec-10" daemon group="main" prio=5 tid=30 nid=30 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown Source)
    at java.util.concurrent.LinkedBlockingQueue.poll(Unknown Source)
    at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
    at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Scheduled executor 5" group="main" prio=5 tid=39 nid=39 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown Source)
    at java.util.concurrent.DelayQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"NuGet Packages Version Checker 1" daemon group="main" prio=5 tid=74 nid=74 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown Source)
    at java.util.concurrent.DelayQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Scheduled executor 1" group="main" prio=5 tid=32 nid=32 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown Source)
    at java.util.concurrent.DelayQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Store issues Spool Thread" daemon group="main" prio=5 tid=44 nid=44 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at java.lang.Thread.sleep(Native Method)
    at net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:620)
    at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:66)
    at net.sf.ehcache.store.DiskStore$SpoolAndExpiryThread.run(DiskStore.java:1074)

"Build estimates updater 1" daemon group="main" prio=5 tid=47 nid=47 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at java.lang.Object.wait(Native Method)
    at jetbrains.buildServer.serverSide.impl.timeEstimation.CachingBuildEstimator$2.run(CachingBuildEstimator.java:2)
    at java.lang.Thread.run(Unknown Source)

"Store indexedBuilds Spool Thread" daemon group="main" prio=5 tid=167 nid=167 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at java.lang.Thread.sleep(Native Method)
    at net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:620)
    at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:66)
    at net.sf.ehcache.store.DiskStore$SpoolAndExpiryThread.run(DiskStore.java:1074)

"Scheduled executor 2" group="main" prio=5 tid=35 nid=35 runnable 
   java.lang.Thread.State: RUNNABLE
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(Unknown Source)
    at java.io.DataInputStream.readFully(Unknown Source)
    at java.io.DataInputStream.readFully(Unknown Source)
    at net.sourceforge.jtds.jdbc.SharedSocket.readPacket(SharedSocket.java:846)
    at net.sourceforge.jtds.jdbc.SharedSocket.getNetPacket(SharedSocket.java:727)
    at net.sourceforge.jtds.jdbc.ResponseStream.getPacket(ResponseStream.java:466)
    at net.sourceforge.jtds.jdbc.ResponseStream.read(ResponseStream.java:103)
    at net.sourceforge.jtds.jdbc.ResponseStream.peek(ResponseStream.java:88)
    at net.sourceforge.jtds.jdbc.TdsCore.wait(TdsCore.java:3932)
    at net.sourceforge.jtds.jdbc.TdsCore.executeSQL(TdsCore.java:1046)
    at net.sourceforge.jtds.jdbc.MSCursorResultSet.cursorClose(MSCursorResultSet.java:919)
    at net.sourceforge.jtds.jdbc.MSCursorResultSet.close(MSCursorResultSet.java:1017)
    at org.apache.commons.dbcp.DelegatingResultSet.close(DelegatingResultSet.java:187)
    at org.apache.commons.dbcp.PoolableConnectionFactory.validateConnection(PoolableConnectionFactory.java:665)
    at org.apache.commons.dbcp.PoolableConnectionFactory.validateObject(PoolableConnectionFactory.java:635)
    at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1198)
    at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:106)
    at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)
    at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111)
    at org.springframework.jdbc.datasource.TransactionAwareDataSourceProxy$TransactionAwareInvocationHandler.invoke(TransactionAwareDataSourceProxy.java:224)
    at $Proxy17.getMetaData(Unknown Source)
    at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.springframework.jdbc.core.JdbcTemplate$CloseSuppressingInvocationHandler.invoke(JdbcTemplate.java:1345)
    at $Proxy17.getMetaData(Unknown Source)
    at jetbrains.buildServer.serverSide.db.ConnectionUtil.detectDatabaseType(ConnectionUtil.java:18)
    at jetbrains.buildServer.serverSide.db.DBFunctions.create(DBFunctions.java:215)
    at jetbrains.buildServer.serverSide.db.DBFunctions.withDB(DBFunctions.java:2267)
    at jetbrains.buildServer.serverSide.db.queries.GenericQuery.underPreparedStatement(GenericQuery.java:469)
    at jetbrains.buildServer.serverSide.db.queries.GenericQuery.doExecuteQuery(GenericQuery.java:401)
    at jetbrains.buildServer.serverSide.db.queries.GenericQuery.access$100(GenericQuery.java:24)
    at jetbrains.buildServer.serverSide.db.queries.GenericQuery$1.run(GenericQuery.java:132)
    at jetbrains.buildServer.serverSide.db.DBFacade$6.doInConnection(DBFacade.java:407)
    at jetbrains.buildServer.serverSide.db.DBFacade$7.doInConnection(DBFacade.java:431)
    at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:341)
    at jetbrains.buildServer.serverSide.db.DBFacade._runSql(DBFacade.java:427)
    at jetbrains.buildServer.serverSide.db.DBFacade.runSql(DBFacade.java:405)
    at jetbrains.buildServer.serverSide.db.queries.GenericQuery.execute(GenericQuery.java:135)
    at jetbrains.buildServer.a.f.k(f.java:46)
    at jetbrains.buildServer.a.m.isMaxNumberOfAuthorizedAgentsExceeded(m.java:8)
    at jetbrains.buildServer.a.k.b(k.java:10)
    at jetbrains.buildServer.a.k.a(k.java:35)
    at jetbrains.buildServer.a.k$0.run(k$0.java)
    at jetbrains.buildServer.util.ExceptionUtil$1.run(ExceptionUtil.java:37)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
    at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Ping" daemon group="main" prio=5 tid=49 nid=49 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at java.lang.Object.wait(Native Method)
    at jetbrains.buildServer.serverSide.impl.BuildServerRunner$BuildServerWorker.run(BuildServerRunner.java:35)
    at java.lang.Thread.run(Unknown Source)

"http-bio-8088-exec-29" daemon group="main" prio=5 tid=4 570 nid=4 570 runnable 
   java.lang.Thread.State: RUNNABLE
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(Unknown Source)
    at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:516)
    at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:501)
    at org.apache.coyote.http11.Http11Processor.setRequestLineReadTimeout(Http11Processor.java:172)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:912)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:539)
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:300)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"http-bio-8088-exec-9" daemon group="main" prio=5 tid=29 nid=29 runnable 
   java.lang.Thread.State: RUNNABLE
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(Unknown Source)
    at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:516)
    at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:501)
    at org.apache.coyote.http11.Http11Processor.setRequestLineReadTimeout(Http11Processor.java:172)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:912)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:539)
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:300)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Low prio executor 3" daemon group="main" prio=4 tid=115 nid=115 waiting 
   java.lang.Thread.State: WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
    at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Agent Distribution Monitor 1" group="main" prio=5 tid=56 nid=56 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown Source)
    at java.util.concurrent.DelayQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Change Observer 1" group="main" prio=5 tid=33 nid=33 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown Source)
    at java.util.concurrent.DelayQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Statistics publisher 1" group="main" prio=5 tid=166 nid=166 waiting 
   java.lang.Thread.State: WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
    at java.util.concurrent.DelayQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"Normal executor 2" group="main" prio=5 tid=57 nid=57 waiting 
   java.lang.Thread.State: WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(Unknown Source)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
    at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

"http-bio-8088-Acceptor-0" daemon group="main" prio=5 tid=17 nid=17 runnable 
   java.lang.Thread.State: RUNNABLE
    at java.net.PlainSocketImpl.socketAccept(Native Method)
    at java.net.PlainSocketImpl.accept(Unknown Source)
    at java.net.ServerSocket.implAccept(Unknown Source)
    at java.net.ServerSocket.accept(Unknown Source)
    at org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(DefaultServerSocketFactory.java:60)
    at org.apache.tomcat.util.net.JIoEndpoint$Acceptor.run(JIoEndpoint.java:207)
    at java.lang.Thread.run(Unknown Source)

"Store contentCache Spool Thread" daemon group="main" prio=5 tid=34 nid=34 waiting 
   java.lang.Thread.State: TIMED_WAITING
    at java.lang.Thread.sleep(Native Method)
    at net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:620)
    at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:66)
    at net.sf.ehcache.store.DiskStore$SpoolAndExpiryThread.run(DiskStore.java:1074)

"Finalizer" daemon group="system" prio=8 tid=3 nid=3 waiting 
   java.lang.Thread.State: WAITING
    at java.lang.Object.wait(Native Method)
    at java.lang.ref.ReferenceQueue.remove(Unknown Source)
    at java.lang.ref.ReferenceQueue.remove(Unknown Source)
    at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)


Memory usage:
   Code Cache: used = 17.03Mb committed = 17.15Mb max = 48.0Mb used/max = 35,49%
   PS Eden Space: used = 50.07Mb committed = 115.43Mb max = 168.56Mb used/max = 29,71%
   PS Survivor Space: used = 724.37Kb committed = 768.0Kb max = 768.0Kb used/max = 94,32%
   PS Old Gen: used = 85.05Mb committed = 128.0Mb max = 341.37Mb used/max = 24,92%
   PS Perm Gen: used = 79.31Mb committed = 79.37Mb max = 270.0Mb used/max = 29,38%
   Total: used = 232.19Mb max = 828.68Mb used/max = 28,02%


Anything else you need?`

\\Stian.
0
Avatar
Stian Fagereng
Created

Could this be the reason, if so what is the solution?















                  
 
                  

                    
                    
                      

                        
                        

                          0
                        

                        
                      

                    
                  
 
                
 
              
                

                  

                    

                      

                        
                        
                            Avatar
                        
                      

                      

                          

                            
                              Stian Fagereng
                            
                          

                          

                            

                              Created 
                            

                          

                      

                    

                    

                      

                        
                      

                    

                  
 
                  

                    
                    
                    
                  

                  

                    
bump.

                  
 
                  

                    
                    
                      

                        
                        

                          0
                        

                        
                      

                    
                  
 
                
 
              
                

                  

                    

                      

                        
                            
                        
                        
                            Avatar
                        
                      

                      

                          

                            
                              Pavel Sher
                            
                          

                          

                            

                              Created 
                            

                          

                      

                    

                    

                      

                        
                      

                    

                  
 
                  

                    
                    
                    
                  

                  

                    
Please enable debug logging on Administration -> Diagnostics page (debug-general), then wait for minute and attach here teamcity-server.log file or send it to teamcity-feedback@jetbrains.com

Also please add several thread dumps taken during that time.

                  
 
                  

                    
                    
                      

                        
                        

                          0
                        

                        
                      

                    
                  
 
                
 
              
                

                  

                    

                      

                        
                        
                            Avatar
                        
                      

                      

                          

                            
                              Stian Fagereng
                            
                          

                          

                            

                              Created 
                            

                          

                      

                    

                    

                      

                        
                      

                    

                  
 
                  

                    
                    
                    
                  

                  

                    
Sorry for the delay...

Here are som logs, hope it's sufficient.


threaddumps


Attachment(s):
threadDump-2012-06-29_08.51.27.txt.zip
threadDump-2012-06-29_09.28.02.txt.zip
threadDump-2012-06-29_09.22.53.txt.zip

                  
 
                  

                    
                    
                      

                        
                        

                          0
                        

                        
                      

                    
                  
 
                
 
              
                

                  

                    

                      

                        
                        
                            Avatar
                        
                      

                      

                          

                            
                              Stian Fagereng
                            
                          

                          

                            

                              Created 
                            

                          

                      

                    

                    

                      

                        
                      

                    

                  
 
                  

                    
                    
                    
                  

                  

                    
another log


Attachment(s):
teamcity-xmlrpc.log.zip

                  
 
                  

                    
                    
                      

                        
                        

                          0
                        

                        
                      

                    
                  
 
                
 
              
                

                  

                    

                      

                        
                        
                            Avatar
                        
                      

                      

                          

                            
                              Stian Fagereng
                            
                          

                          

                            

                              Created 
                            

                          

                      

                    

                    

                      

                        
                      

                    

                  
 
                  

                    
                    
                    
                  

                  

                    
and.....


Attachment(s):
teamcity-vcs.log.zip

                  
 
                  

                    
                    
                      

                        
                        

                          0
                        

                        
                      

                    
                  
 
                
 
              
                

                  

                    

                      

                        
                        
                            Avatar
                        
                      

                      

                          

                            
                              Stian Fagereng
                            
                          

                          

                            

                              Created 
                            

                          

                      

                    

                    

                      

                        
                      

                    

                  
 
                  

                    
                    
                    
                  

                  

                    
and......made rar for the last three....


Attachment(s):
TeamCity Loggers.rar.zip

                  
 
                  

                    
                    
                      

                        
                        

                          0
                        

                        
                      

                    
                  
 
                
 
              
                

                  

                    

                      

                        
                        
                            Avatar
                        
                      

                      

                          

                            
                              Stian Fagereng
                            
                          

                          

                            

                              Created 
                            

                          

                      

                    

                    

                      

                        
                      

                    

                  
 
                  

                    
                    
                    
                  

                  

                    
neat feature that you auto-zip uploading txt files. :)

But I did not manage to upload a txt file of 10mb. Had to rar it...

                  
 
                  

                    
                    
                      

                        
                        

                          0
                        

                        
                      

                    
                  
 
                
 
              
                

                  

                    

                      

                        
                        
                            Avatar
                        
                      

                      

                          

                            
                              Stian Fagereng
                            
                          

                          

                            

                              Created 
                            

                          

                      

                    

                    

                      

                        
                      

                    

                  
 
                  

                    
                    
                    
                  

                  

                    
bump - bump.

                  
 
                  

                    
                    
                      

                        
                        

                          0
                        

                        
                      

                    
                  
 
                
 
              
                

                  

                    

                      

                        
                            
                        
                        
                            Avatar
                        
                      

                      

                          

                            
                              Yegor Yarko
                            
                          

                          

                            

                              Created 
                            

                          

                      

                    

                    

                      

                        
                      

                    

                  
 
                  

                    
                    
                    
                  

                  

                    
Stian,

Sorry for the delay...

It seems that TeamCity is hanging trying to communicate with the database.

If still actual, please try to restart TeamCity server (and may be also database) as a remedy.

If this issue reproduces, please let us know so that we can suggest a way to collect debugging information for the case.

                  
 
                  

                    
                    
                      

                        
                        

                          0
                        

                        
                      

                    
                  
 
                
 
              
                

                  

                    

                      

                        
                        
                            Avatar
                        
                      

                      

                          

                            
                              Stian Fagereng
                            
                          

                          

                            

                              Created 
                            

                          

                      

                    

                    

                      

                        
                      

                    

                  
 
                  

                    
                    
                    
                  

                  

                    
It does, so we had to add a job that restarts teamcity each morning. 

So a solution would be nice. :)

                  
 
                  

                    
                    
                      

                        
                        

                          0
                        

                        
                      

                    
                  
 
                
 
              
                

                  

                    

                      

                        
                            
                        
                        
                            Avatar
                        
                      

                      

                          

                            
                              Leonid
                            
                          

                          

                            

                              Created 
                            

                          

                      

                    

                    

                      

                        
                      

                    

                  
 
                  

                    
                    
                    
                  

                  

                    
Hello Stian,

Let me describe what may couse such problem.

The most probably case is that MS SQL server session crashes during some SQL query execution. I experienced such problem with MS SQL 2005 and 2008 two years ago.

In this case one of TeamCity's SQL query causes an interanl error in MS SQL server session that is not handled so the session crashes. MS SQL TCP listener, however, is not nitified (about this crash) and continuing to wait the session response. TeamCity waits for the response of TCP connection
too. It causes that a part of TeamCity hangs.

As of what can we do in this case.

1. Try to specify a connection timeout. Use the 'socketTimeout' option (described here: http://jtds.sourceforge.net/faq.html). It requires also to verify connection at each borrowing from the pool (in other case other TeamCity parts wil crash). 
In order to do this, add the folloing properties:
connectionProperties.socketTimeout=3600
 testOnBorrow=true
However, TeamCity can use long queries during cleanup, so ensure that this timeout is longer that the cleanup duration.

When such timeout is catched the problematic query should be logged so we'd get ability to refine/tune this query.

2. You can try to use the native JDBC driver instead of the JTDS one. It's described here:
http://confluence.jetbrains.net/display/TCD7/Setting+up+an+External+Database#SettingupanExternalDatabase-Nativedriver
It can help if the problem caused by a JTDS specifics.


Please let me know did something help.


Best regards
Leonid

                  
 
                  

                    
                    
                      

                        
                        

                          0
                        

                        
                      

                    
                  
 
                
 
              
                

                  

                    

                      

                        
                        
                            Avatar
                        
                      

                      

                          

                            
                              Stian Fagereng
                            
                          

                          

                            

                              Created 
                            

                          

                      

                    

                    

                      

                        
                      

                    

                  
 
                  

                    
                    
                    
                  

                  

                    
Thanks for your reply.

I would like to test to add the 
connectionProperties.socketTimeout=3600
testOnBorrow=true



options to the db connection settings. But beeing a while since I did the migration I did not manage to get where the dbconnection settings is located.
I did read the faq you linked to. :)

\\Stian.

                  
 
                  

                    
                    
                      

                        
                        

                          0
                        

                        
                      

                    
                  
 
                
 
              

              
            
 
          

          

            
Please sign in to leave a comment.