How to test a TeamCity Upgrade for a larger installation

Over the last year or two, we have grown more and more reliant on TeamCity for all sorts of internal services from building and testing to updating various web sites. Which is fine in itself, as TC so far as proven quite stable.

But.... that also makes it more difficult to "just" upgrade TeamCity when a new version is released as all sorts of stuff might not work anymore. Which leads me to the question:

       How do you out there in userland test a new TeamCity version before it is deployed to production?

A little background:

We currently have ~100 different types of build configurations. These are used to build 4 different products in various versions and configurations, which currently adds up to ~900 active build configurations and ~130 active VCS roots. We have ~40 agents of various types, some generic for compilations and other common stuff, some more specialized with custom hardware attached typically used for tests.

Much as I would like, I cannot get a complete replication of the current system to test an upgrade of TeamCity. I have hardware resources for 10 agents so I can test most of the build configurations - one-by-one.

But can we use the product licenses in such a test system? Or are we reduced to the 3 agent limit of the professional edition?

How do you usually cope with the set-up of the test system? We usually make a copy of the database and then we have a sea full of user names and passwords, host names, services, etc - most of these are kept in places where we can change them manually, but... there must be a better way.

I'm very interested in hearing from people that have similar problems and how you have streamlines the test process.

1 comment
Comment actions Permalink

Hi Tonny, thanks for your write up. I'd also like to see whether other users can provide a look into how they test-upgrade their systems. Automating this process is obviously a complicated task and usually largely dependent on workflows, but having some feedback on the methods you and others follow is always useful in considering how to improve the system.

We have a generic suggestion in our documentation: https://confluence.jetbrains.com/pages/viewpage.action?pageId=74845225#HowTo...-Test-driveNewerTeamCityVersionbeforeUpgrade , simply create a side test server, upgrade and test most of the stuff in it. Licensing-wise, the easiest option is to request an evaluation license from our sales, that should give you license coverage for a long enough period to test all functionalities you need from the newer release.

1

Please sign in to leave a comment.