Migration from 2017.1.3 to 2018.1 not possible? Backup files not supported.

So I backed up the teamcity configuration that I have set up over a long time.

Then I upgraded to the 2018.1 version, tried to restore the backup and was greeted with the following message:

The selected backup file version (829) does not match the current version of the TeamCity database (856). Only backup files created with a TeamCity server of the same version as the current server are supported.

I hope this is not another way of saying that my backup is useless and I need to start from 0.
Also I do not care so much about the build logs, but rather about all the build configurations etc.

Please advise, whether migration is possible or not. And if it's possible how would I go about it?

Thanks you for your advice,
Prasath

3
11 comments

Also I am using the Docker images. Previously I used the 2017.1.3 docker image and now the 2018.1 docker image. There is no way I can just click "update" from the old version.

0

Hi Prasath,

the problem here is slightly different to what you mention. If you follow the upgrade process, after replacing the image, you should have an image of the new version (2018.1, file version 856) and a data directory and database of the old version (2017.1.3, with version 829). At this point, TeamCity upgrades data directory and database to the newest version at the same time. If they aren't both upgraded, they should be rolled back.

The mismatch happens when the database or the data directory have been upgraded with one installation but the other hasn't, which shouldn't happen with the regular installation. In your case, the database was upgraded but the data directory wasn't.

Once the problem has kicked in, the usual solution is to restore the already-updated part (in this case the database) to a copy of the older release and then proceed again with the upgrade.

1

Hi Prasath,

Whats your Database, whats the size of your DB and which OS is Teamcity is deployed?

Do you have docker compose file, please share the details.

 

Thank you.

0

Hi Denis,

what is the difference between data directory and database.

Thank you.

0

Hi Prashanth,

I'm not sure I understand your question. I'm going to answer it explicitly but I really feel like you are asking a different thing that I'm missing.

The data directory is a... directory, where TeamCity stores its configuration options, caches, artifacts, plugins, etc. The database is... a database, by default an integrated HSQLDB that should never be used in production. In the database TeamCity stores information about users, builds, tests, vcs roots, etc. About their contents. Their configuration is stored in the data directory.

The data directory is kept as a directory as it makes making changes to it very easy, which would be a bit more complex if they needed to be done through the database. Because they carry related information, though, they need to be kept in sync. If TeamCity detects they are not using the same version, this might be due to incomplete backup restores, wrong database or data directory passed to the TeamCity server, etc, and thus TeamCity cannot be started safely.

Hope this makes sense. If you had a different question please don't hesitate to expand.

0

Hi Denis, last couple of days I was involved in upgrading TeamCity successfully. But unfortunately, still no luck here. Let me explain the two different approaches and how each attempt fails with a different error message.

Attempt 1: Use TeamCity built-in backup mechanism

This is actually a very cool feature. Very pleasant user experience. You create a backup from the UI and you can simply select the parts you want to backup such as database, logs, configs, etc. I had a few use cases where a restored such a backup to a new install. Some things like connections at the root level disappeared, but all the project configurations and database entries came back up. The built-in backup works if you remain in the same version of TeamCity. Basically, you are not allowed to upgrade if you use Docker.

This is what happens if you upgrade you docker images to the 2018.1 or 2018.1.1 version: You spin up a new install and go through "Server Administration" and then "Projects Import" process. You are asked to upload the backup zip archive and after you do that you are greeted with this error message:

The selected backup file version (829) does not match the current version of the TeamCity database (856). Only backup files created with a TeamCity server of the same version as the current server are supported.

@Denis, I am not sure how to get to the point where this happens: "At this point, TeamCity upgrades data directory and database to the newest version at the same time. If they aren't both upgraded, they should be rolled back." Seems like the above approach "Server Administration" > "Projects Import" > "Upload Archive" is not the approach you are talking about. Can you explain in more detail how to get to this point?


Attempt 2: Backup individual folders manually

I kind of was frustrated with the backup restore process. Going through readme and documentation some TC staff recommends to do a manual backup of just the parts you care about if you have more complex backup requirements. In my case, I have the simplest backup criterion possible, just simple version upgrade. Anyways, I worked on a way to backup volumes to AWS S3. That's what I did for the manual backup. I was thinking about what are the important pieces that I care about? Answer: "Build configurations". So I decided to backup the "/data/teamcity_server/datadir/config" directory. I did not care about build logs and database entries. I am ok if the database starts at zero. But please let me take over my build configurations. In any case, I managed to backup that directory. I wiped out the existing docker container. Spin up a new container using the new image pointing to a volume with the old data. I was filled with hope that TeamCity will show a popup whether or not I want to upgrade the data directory. Unfortunately, it did not happen. Here's what I am greeted with this time. What's even more frustrating is the part that comes up for the next step: "not defined yet".

Data parts are inconsistent: the Data Directory exists (from another version of TeamCity) but the database does not.

Current Startup State

Startup status

Current step: TeamCity server startup error 
Next step: not defined yet 

Data Directory

Directory path: /data/teamcity_server/datadir exists

Database properties file exists

Internal database file: not found

Database

Database type: POSTGRESQL

Database connection URL: jdbc:postgresql://teamcity-postgres.default.svc.cluster.local/teamcity

JDBC driver version: 42.1 (PostgreSQL JDBC Driver)

Database system version: 9.6.7 (PostgreSQL)

Versions

Software version: 856

Data directory version: 829

Database version: unknown

Logs

Logs path: /opt/teamcity/logs

 

I am sorry for the negative mood way in which I describe these events. But upgrading has been a horrible experience for me which knocked me out for more than a week now. I am looking forward for good news and more details on how I can get the upgrade done successfully. Thank you :)

 

0

> But upgrading has been a horrible experience for me which knocked me out for more than a week now

Here-here. Trying to migrate from a poorly setup on-site windows machine to a cloud-based docker machine has been really frustraiting. Then I got my docker-compose file working with 2017.2.2 (the version that was on windows), get everything running, then wipe away my images, update my docker file to use 2018.1.1, go through the import process again, and I get this message.

So I guess what I'm supposed to do is simply replace the image-version in-place?

I would love it if the teamcity team would invest some time here. There's a fair bit I like about teamcity, but this process has been far from painless.

0

I also want to deploy TeamCity server in Windows container and have stuck with this error message.

The server works good but I cannot export/import a project.

BTW I've found the statement here https://confluence.jetbrains.com/display/TCD18/Restoring+TeamCity+Data+from+Backup

"Backups created with TeamCity 6.0+ can be restored using the same or more recent TeamCity versions."

Does it mean that performing full restore (all projects) will work correctly?

 

0

And one more question.

If 2017th version of container image is deployed already and works correctly, can I stop the container and simply load the 2018th image and start it?

Will 2018th server automatically update all necessary data/db information on start?

 

0

Sorry all for not having answered earlier. Unfortunately the notification of answers arrived while i was out and got buried in emails.

 

@Prasath: You don't need to restore the backup after the upgrade. Much less use the "Projects Import", which is a different feature for an unrelated task. The upgrade process should upgrade the data and all the data should be retained if you simply pass the same data directory to the new image.

Upgrading using docker is perfectly possible and needs no special treatment, as explained here: https://confluence.jetbrains.com/display/TCD18/Upgrade#Upgrade-UpgradingTeamCitystartedfromDockerimages

Auto upgrade (built in) will not work because a docker image will not be modified by it, only the running process. Once the VM is shut down, the image will be restored to the last version.

Backups aren't meant to be restored after the upgrade happens. Backup should *not* be restored unless you are reverting to the old version.

 

@All: Simply replacing the image with the new one will replace the folders used by the TeamCity server with the new version. Upon startup, if you are passing the old teamcity data directory kept external to the images, which you should be doing, it will detect that both the data directory and the database belong to the old version and will try to update both. If the process fails for some reason, the changes should be rolled back in both places with the backup.

The errors happening through this thread are of a mismatch (or one of the two entirely missing) between the two versions, that should only happen when the data directory and the database aren't of the same version. For example, if you are trying to restore the backup of an old version of the data directory but not restoring the database which has been upgraded during the process.

May I ask you all why are you trying to restore a backup after the upgrade has finished?

 

@Denis: Yes, that is the expected process. Upon startup after replacing the image, you will access the UI and will be prompted for administrator access (a token from the logs). Then you will have the option to upgrade the data directory and database and once the process finishes it should move on just fine.

0

"restore a backup after the upgrade has finished?" 

In our case the upgrade of main TS server is in process. I guess there are some license compatibility issues with 2018 version.

 

Before that I simply want to use TeamCity docker image since it is considered (can be wrong) as more lightweight and flexible way to deploy (comparing to VM).

To do so I first need to insure it works as expected with all important features that we use currently. Therefore I need to setup kinda clone of current teamcity server which we use in our small team. The team is small but number of build steps and other features is not :)

I exported target projects from current TeamCity server and tried to import them into cloned one.

BTW I already did it once but both container and current server were of the same 2017.* version.

Today I decided to deploy container from scratch on another host machine and seems like downloaded the last container image which is 2018. Then I've met the error while being importing target project.

 

Ok, I will try to just pull 2017 container image, import project there and then pull newer 2018 image.

Thanks

 

0

Please sign in to leave a comment.