Teamcity is very slow (High CPU usage)

Teamcity was installed as war application in september. It work good, but for last time it work very slow with high CPU usage. It need very long time for start:
# more teamcity-server.log | grep "TeamCity Spring context created"
[2013-09-10 09:23:45,385]   INFO -  jetbrains.buildServer.STARTUP - TeamCity Spring context created in: 18453 ms
[2013-09-12 14:42:21,289]   INFO -  jetbrains.buildServer.STARTUP - TeamCity Spring context created in: 20189 ms
[2013-09-12 14:59:31,069]   INFO -  jetbrains.buildServer.STARTUP - TeamCity Spring context created in: 18316 ms
[2013-09-13 06:26:40,164]   INFO -  jetbrains.buildServer.STARTUP - TeamCity Spring context created in: 18086 ms
[2013-12-02 10:12:11,464]   INFO -  jetbrains.buildServer.STARTUP - TeamCity Spring context created in: 21558 ms
[2013-12-19 11:34:10,359]   INFO -  jetbrains.buildServer.STARTUP - TeamCity Spring context created in: 376071 ms
[2014-01-03 16:40:27,225]   INFO -  jetbrains.buildServer.STARTUP - TeamCity Spring context created in: 1038596 ms
[2014-01-06 10:13:24,754]   INFO -  jetbrains.buildServer.STARTUP - TeamCity Spring context created in: 431689 ms
[2014-01-06 10:48:58,246]   INFO -  jetbrains.buildServer.STARTUP - TeamCity Spring context created in: 378856 ms
[2014-01-06 14:08:41,377]   INFO -  jetbrains.buildServer.STARTUP - TeamCity Spring context created in: 449685 ms
[2014-01-06 15:29:52,936]   INFO -  jetbrains.buildServer.STARTUP - TeamCity Spring context created in: 363044 ms

What  happened with teamcity?

2 comments
Comment actions Permalink

I enable debug log and found in server log message every second:
[2014-01-06 18:00:21,792]  ERROR - erverSide.search.SearchService - SearchService.enqueueHistory
java.io.FileNotFoundException: no segments* file found in org.apache.lucene.store.NIOFSDirectory@/vol/teamcity/system/caches/search: files:
        at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:655)
        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:69)
        at org.apache.lucene.index.IndexReader.open(IndexReader.java:476)
        at org.apache.lucene.index.IndexReader.open(IndexReader.java:314)
        at jetbrains.buildServer.serverSide.search.SearchService.getIndexSearcher(SearchService.java:439)
        at jetbrains.buildServer.serverSide.search.SearchService.enqueueHistory(SearchService.java:578)
        at jetbrains.buildServer.serverSide.search.BackgroundIndexer.run(BackgroundIndexer.java:26)
        at java.lang.Thread.run(Thread.java:679)
[2014-01-06 18:00:21,815]  DEBUG -   jetbrains.buildServer.SERVER - BackgroundIndexer.run in 0ms
[2014-01-06 18:00:22,394]  DEBUG - ains.buildServer.util.WatchDog - Flush Queue start: 0 msec
[2014-01-06 18:00:22,534]  DEBUG - ains.buildServer.util.WatchDog - Flush Queue done: 113 msec
[2014-01-06 18:00:22,828]   INFO -   jetbrains.buildServer.SERVER - Looking for not yet indexed builds in history...
[2014-01-06 18:00:22,852]  ERROR - erverSide.search.SearchService - SearchService.enqueueHistory
java.io.FileNotFoundException: no segments* file found in org.apache.lucene.store.NIOFSDirectory@/vol/teamcity/system/caches/search: files:
        at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:655)
        at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:69)
        at org.apache.lucene.index.IndexReader.open(IndexReader.java:476)
        at org.apache.lucene.index.IndexReader.open(IndexReader.java:314)
        at jetbrains.buildServer.serverSide.search.SearchService.getIndexSearcher(SearchService.java:439)
        at jetbrains.buildServer.serverSide.search.SearchService.enqueueHistory(SearchService.java:578)
        at jetbrains.buildServer.serverSide.search.BackgroundIndexer.run(BackgroundIndexer.java:26)
        at java.lang.Thread.run(Thread.java:679)
[2014-01-06 18:00:22,861]  DEBUG -   jetbrains.buildServer.SERVER - BackgroundIndexer.run in 0ms

0

Please sign in to leave a comment.