We have a large series of JUnit runs, that generates couple of gigs of build logs and lasts about 12-13hrs, reporting results for about ~80000 tests.
When we come in in the morning, after the test runs are finished, the TeamCity server is very very slow, sometimes taking longer the 2 minutes to load a individual project page. Restarting the TeamCity server causes the issue to go away.
It also appears that the tests runs start to get slower as the evening goes on.
I have uploaded a profile snapshot, taken when the server was running super slow, to ftp://ftp.intellij.net/.uploads/slow_after_night_of_junits.snapshot.
I figure part of the problem is all the data we dump to the console/logs, and I am working on tweaking that so we dump less. But even then, it shouldn't slow down the webapp like this.