TeamCity 2.0 unable to recover from SubVersion outage

Hi,

Our SVN repo went down for about 2 hours, out of disk. Unfortunately, SVN and TeamCity are on the same server, so TC hung too. When SVN came back up, TeamCity was still hung. We restarted TC a few times, but it appeared to have so many builds pending that it just hung again after few seconds.

We were finally able to get it going by moving nearly all of the (about 40) project config dirs out of .BuildServer/config to a tmp dir, starting TC, and feeding the config dirs slowly back in as builds began going through.

Is there a load test on your side that would cover this scenario? Has it been fixed in later versions?

Thanks!
-chris

2 comments

Chris,

We don't have such a load test, it worth adding it. Do you have probably some logs when TeamCity was hanged?

Kind regards,
KIR

0

Hey Kirill,

Sorry, this won't help you much. I was on vacation (Niagara Falls with the family, very cool!) and the logs are not there any more. I did take a look at them when we were in the middle of it, and didn't notice anything explicitly revealing. A trained eye might have, though. I'll save 'em off if it happens again.

-chris

0

Please sign in to leave a comment.