Over the last year or two, we have grown more and more reliant on TeamCity for all sorts of internal services from building and testing to updating various web sites. Which is fine in itself, as TC so far as proven quite stable.
But.... that also makes it more difficult to "just" upgrade TeamCity when a new version is released as all sorts of stuff might not work anymore. Which leads me to the question:
How do you out there in userland test a new TeamCity version before it is deployed to production?
A little background:
We currently have ~100 different types of build configurations. These are used to build 4 different products in various versions and configurations, which currently adds up to ~900 active build configurations and ~130 active VCS roots. We have ~40 agents of various types, some generic for compilations and other common stuff, some more specialized with custom hardware attached typically used for tests.
Much as I would like, I cannot get a complete replication of the current system to test an upgrade of TeamCity. I have hardware resources for 10 agents so I can test most of the build configurations - one-by-one.
But can we use the product licenses in such a test system? Or are we reduced to the 3 agent limit of the professional edition?
How do you usually cope with the set-up of the test system? We usually make a copy of the database and then we have a sea full of user names and passwords, host names, services, etc - most of these are kept in places where we can change them manually, but... there must be a better way.
I'm very interested in hearing from people that have similar problems and how you have streamlines the test process.