Based on your advice (in a previous thread), I want to upgrade my company's production TC enterprise server from 6.0.3 to 7.1.4. We would like to take advantage of all the new enhancements that JetBrains has included in this version.
The problem is that the last two times my colleagues tried to upgrade (to 7.1 back in Sept), we had to rollback because, in the new version, it was taking a lot longer to check for changes. Initial testing of the upgrade seemed to indicate (we’re not sure though) that this was a capacity-related issue due to the number of VCS roots in our instance.
To be clear, the upgrade itself did not fail, just the performance in the new version was terribly slow. Therefore, we decided to stick with 6.0 until recently when the performance is degrading our developers have not been able to take advantage of the new features that TC offers in 7.1.4.
I would like to get your help in possibly determining the cause behind the slowness as well as get your advice on how to proceed with the upgrade this time. So, there are actually two threads here:
1 - Last Upgrade/Rollback – root cause:
Unfortunately, during the last upgrade attempts there were no thread dumps taken while testing. However, I have attached some of the relevant parts of the logs (we tried upgraded on 9/10 and 9/17). My hope is that you may be able to see something that we may have missed.
2 – This attempt – advice:
In order to avoid breaking production this time I would like to test out the upgrade on QA. My hope is that if I can get QA as close to prod as possible, I can test out most of the possible breaks, thus minimizing the risk on prod. Currently, we have 24 build agents on production and we use SVN as our VCS. We also use MS SQL as our external DB. I can replicate the DB, install the new server on QA and install a new build agent that uses the new server. However, replicating SVN will be a problem.
My questions around this are –
- How do you test performance related issues? What are some of the smoke tests you use in QA?
- How do I replicate the “checking for changes” part without having to replicate prod SVN or having to point the prod SVN to the QA version.
- What are some steps that I can do in QA to ensure that this time the upgrade on prod does not fail
- Am I approaching this wrong? Should I be following a different route.
- Finally, does JetBrains assist in upgrades/support (remote assist possibly)? If so, this is another avenue we can explore to guardrail against failing in prod. What are the costs associated with this kind of support?
Thanks in advance for your time.