Build History Clean-up taking very long time

We're using TC 5.1.3 and have the Build History clean up enabled to run once a day (midnight).

The problem is it's taking 6 hours to run, which seems an extremely long time and results in all the builds scheduled during that period to be queued up and only run when it finishes.

Our default clean up policy is all artefacts 30 days older than the last successful build and history kept forever.

Some other stats on our usage:
139 active projects with 860 build configurations
Around 300GB of artefacts (stable)
TC running on a physical machine with 3gb RAM, Intel P4 3.4GHz Processor, Win2003 server

Please can someone let me know if that sounds normal?

Thanks.

4 comments
Comment actions Permalink

Hi Rob

Could you please post teamcity-server.log, it should help to identify the bottle neck.

Thanks

0
Comment actions Permalink

Hi Michael,

Log file is attached. Here is an extract which shows logs stoppping at 00:06 until 07:19. Suggests a problem with "Cleaning exensions data"?

Thanks,

Rob

[2012-01-30 00:05:00,955]   INFO -  jetbrains.buildServer.CLEANUP - Performing cleanup EVERYTHING for 54 builds
[2012-01-30 00:05:00,955]   INFO -  jetbrains.buildServer.CLEANUP - Cleaning extensions data...
[2012-01-30 00:05:03,377]   INFO -  jetbrains.buildServer.CLEANUP - Deleting build logs...
[2012-01-30 00:05:06,517]   INFO -  jetbrains.buildServer.CLEANUP - Cleaning build history data from the database...
[2012-01-30 00:05:07,033]   INFO -  jetbrains.buildServer.CLEANUP - Deleting build promotions: [518090, 516515, 518461, 516870, 518452, 518453, 518319, 517222, 518321, 518206, 518444, 518099, 518435, 518434, 518333, 518100, 515462, 518153, 518275, 518425, 518158, 515988, 518416, 518417, 518287, 518168, 518288, 518409, 518402, 518299, 518401, 518015, 518369, 518380, 518242, 518379, 518389, 518266, 518267, 518042, 518041, 518258, 518341, 518343, 516314, 518474, 517937, 518473, 518235, 518354, 517948, 518363, 517947, 518362]
[2012-01-30 00:05:07,142]   INFO -  jetbrains.buildServer.CLEANUP - Cleaning build statistics history data from the database...
[2012-01-30 00:05:07,174]   INFO -  jetbrains.buildServer.CLEANUP - Deleting build artifacts...
[2012-01-30 00:06:47,387]   INFO -  jetbrains.buildServer.CLEANUP -  Deleting artifacts directory: c:\teamcity\.BuildServer\system\artifacts\SevenDigital.Account\Master - 2. Code Metrics (last succesful)\506146
[2012-01-30 00:06:47,794]   INFO -  jetbrains.buildServer.CLEANUP -  Deleting artifacts directory: c:\teamcity\.BuildServer\system\artifacts\SevenDigital.API\Master - 1a. Code Metrics (last successful) (auto)\505510
[2012-01-30 00:06:48,106]   INFO -  jetbrains.buildServer.CLEANUP - Performing cleanup ARTIFACTS for 219590 builds
[2012-01-30 00:06:48,122]   INFO -  jetbrains.buildServer.CLEANUP - Cleaning extensions data...
[2012-01-30 07:19:07,000]   INFO -  jetbrains.buildServer.CLEANUP - Deleting build artifacts...
[2012-01-30 07:21:54,527]   INFO -  jetbrains.buildServer.CLEANUP -  Deleting artifacts directory: c:\teamcity\.BuildServer\system\artifacts\DatabaseMigraine\master - 01 - Solution build\466090
[2012-01-30 07:21:54,621]   INFO -  jetbrains.buildServer.CLEANUP -  Deleting artifacts directory: c:\teamcity\.BuildServer\system\artifacts\DatabaseMigraine\master - 01 - Solution build\467173
[2012-01-30 07:21:54,714]   INFO -  jetbrains.buildServer.CLEANUP -  Deleting artifacts directory: c:\teamcity\.BuildServer\system\artifacts\SevenDigital\SevenDigital - 1 - Trunk - Build and Unit Test\466156
[2012-01-30 07:21:54,933]   INFO -  jetbrains.buildServer.CLEANUP -  Deleting artifacts directory: c:\teamcity\.BuildServer\system\artifacts\SevenDigital\SevenDigital - 1 - Trunk - Build and Unit Test\467094
[2012-01-30 07:21:55,105]   INFO -  jetbrains.buildServer.CLEANUP -  Deleting artifacts directory: c:\teamcity\.BuildServer\system\artifacts\SevenDigital.API\Master - 1. Build and Unit Test\465656
[2012-01-30 07:21:55,761]   INFO -  jetbrains.buildServer.CLEANUP -  Deleting artifacts directory: c:\teamcity\.BuildServer\system\artifacts\SevenDigital.API\Master - 1. Build and Unit Test\466314
[2012-01-30



Attachment(s):
teamcity-server.log.zip
0
Comment actions Permalink

Sorry, and the clean up is scheduled to start at midnight (00:00)

0
Comment actions Permalink

Rob,

We need additional details to localize the issue. Please enable debug logging and reproduce it once again.

Thanks

0

Please sign in to leave a comment.