Failed to upgrade TeamCity 2018.2
I recently tried to upgrade my TeamCity server to 2018.2, but it failed and I was forced to roll it back. I'm currently using 2017.2.2 so would like to get this update through.
Below is some logs from the failed upgrade, it does suggest there is something in my database it does not like, but not enough information or me to find it. I could be mistaken though.
[2018-12-07 15:39:42,926] INFO - de.versioning.ConvertPerformer - Converter 861[script:create_test_metadata_tables] finished.
[2018-12-07 15:39:42,926] INFO - de.versioning.ConvertPerformer - Converter 862[script:create_node_tasks_table] started...
[2018-12-07 15:39:42,957] INFO - de.versioning.ConvertPerformer - Converter 862[script:create_node_tasks_table] finished.
[2018-12-07 15:39:42,957] INFO - de.versioning.ConvertPerformer - Converter 863[class:SwitchToNodesResponsibilitiesConverter] started...
[2018-12-07 15:39:42,972] INFO - de.versioning.ConvertPerformer - Converter 863[class:SwitchToNodesResponsibilitiesConverter] finished.
[2018-12-07 15:39:42,972] INFO - de.versioning.ConvertPerformer - Converter 864[class:AddHeartbeatResponsibilitiesConverter] started...
[2018-12-07 15:39:42,988] INFO - de.versioning.ConvertPerformer - Converter 864[class:AddHeartbeatResponsibilitiesConverter] finished.
[2018-12-07 15:39:42,988] INFO - de.versioning.ConvertPerformer - Converter 865[class:MoveCustomDataStorageToDatabaseConverter] started...
[2018-12-07 15:39:43,177] INFO - jetbrains.buildServer.SERVER - Start moving custom data storage files to database...
[2018-12-07 15:39:44,339] ERROR - jetbrains.buildServer.STARTUP - Exception UnknownUpgradeError in stage UPGRADE: Upgrading error: Problem encountered while performing java converter MoveCustomDataStorageToDatabaseConverter
jetbrains.buildServer.maintenance.exceptions.UnknownUpgradeError: Upgrading error: Problem encountered while performing java converter MoveCustomDataStorageToDatabaseConverter
at jetbrains.buildServer.maintenance.StartupProcessor.doUpgrade(StartupProcessor.java:818)
at jetbrains.buildServer.maintenance.StartupProcessor.access$1400(StartupProcessor.java:561)
at jetbrains.buildServer.maintenance.StartupProcessor$3.call(StartupProcessor.java:17)
at jetbrains.buildServer.util.NamedThreadFactory.executeWithNewThreadName(NamedThreadFactory.java:91)
at jetbrains.buildServer.maintenance.StartupProcessor.processConcreteStage(StartupProcessor.java:744)
at jetbrains.buildServer.maintenance.StartupProcessor.processConcreteStageSafe(StartupProcessor.java:224)
at jetbrains.buildServer.maintenance.StartupProcessor.processTeamCityLifecycle(StartupProcessor.java:870)
at jetbrains.buildServer.maintenance.StartupProcessor.access$000(StartupProcessor.java:884)
at jetbrains.buildServer.maintenance.StartupProcessor$1.run(StartupProcessor.java:3)
at java.lang.Thread.run(Thread.java:748)
Caused by: jetbrains.buildServer.serverSide.versioning.ConvertException: Problem encountered while performing java converter MoveCustomDataStorageToDatabaseConverter
at jetbrains.buildServer.serverSide.versioning.ConvertPerformer.runConverterStepJavaClass(ConvertPerformer.java:204)
at jetbrains.buildServer.serverSide.versioning.ConvertPerformer.runConverterSteps(ConvertPerformer.java:161)
at jetbrains.buildServer.serverSide.versioning.ConvertPerformer.runConverter(ConvertPerformer.java:111)
at jetbrains.buildServer.serverSide.versioning.ConvertPerformer.runConverters(ConvertPerformer.java:87)
at jetbrains.buildServer.serverSide.versioning.VersionManager.upgradeIt(VersionManager.java:381)
at jetbrains.buildServer.maintenance.StartupProcessor$UpgradePerformer.doUpgrade(StartupProcessor.java:3)
at jetbrains.buildServer.maintenance.StartupProcessor.lambda$doUpgrade$7(StartupProcessor.java:751)
at jetbrains.buildServer.serverSide.db.TeamCityDatabaseManager$3.run(TeamCityDatabaseManager.java:942)
at jetbrains.buildServer.serverSide.db.TeamCityDatabaseManager$3.run(TeamCityDatabaseManager.java:937)
at jetbrains.buildServer.serverSide.db.TeamCityDatabaseManager.withMaintenanceDBF(TeamCityDatabaseManager.java:921)
at jetbrains.buildServer.serverSide.db.TeamCityDatabaseManager.withMaintenanceDBF(TeamCityDatabaseManager.java:937)
at jetbrains.buildServer.maintenance.StartupProcessor.doUpgrade(StartupProcessor.java:940)
... 9 more
Caused by: com.thoughtworks.xstream.io.StreamException:
at com.thoughtworks.xstream.io.xml.XppReader.pullNextEvent(XppReader.java:124)
at com.thoughtworks.xstream.io.xml.AbstractPullReader.readRealEvent(AbstractPullReader.java:148)
at com.thoughtworks.xstream.io.xml.AbstractPullReader.readEvent(AbstractPullReader.java:141)
at com.thoughtworks.xstream.io.xml.AbstractPullReader.move(AbstractPullReader.java:118)
at com.thoughtworks.xstream.io.xml.AbstractPullReader.moveDown(AbstractPullReader.java:103)
at com.thoughtworks.xstream.io.xml.XppReader.<init>(XppReader.java:63)
at com.thoughtworks.xstream.io.xml.AbstractXppDriver.createReader(AbstractXppDriver.java:54)
at com.thoughtworks.xstream.XStream.fromXML(XStream.java:1398)
at com.thoughtworks.xstream.XStream.fromXML(XStream.java:1389)
at jetbrains.buildServer.serverSide.versioning.converters.MoveCustomDataStorageToDatabaseConverter.deserializeObject(MoveCustomDataStorageToDatabaseConverter.java:204)
at jetbrains.buildServer.serverSide.versioning.converters.MoveCustomDataStorageToDatabaseConverter.loadProperties(MoveCustomDataStorageToDatabaseConverter.java:170)
at jetbrains.buildServer.serverSide.versioning.converters.MoveCustomDataStorageToDatabaseConverter.convert(MoveCustomDataStorageToDatabaseConverter.java:103)
at jetbrains.buildServer.serverSide.versioning.ConvertPerformer.runConverterStepJavaClass(ConvertPerformer.java:195)
... 20 more
Caused by: org.xmlpull.v1.XmlPullParserException: only whitespace content allowed before start tag and not r (position: START_DOCUMENT seen r... @1:1)
at org.xmlpull.mxp1.MXParser.parseProlog(MXParser.java:1517)
at org.xmlpull.mxp1.MXParser.nextImpl(MXParser.java:1393)
at org.xmlpull.mxp1.MXParser.next(MXParser.java:1091)
at com.thoughtworks.xstream.io.xml.XppReader.pullNextEvent(XppReader.java:109)
... 32 more
[2018-12-07 15:39:44,371] INFO - jetbrains.buildServer.STARTUP - Current stage: TeamCity server startup error
[2018-12-07 16:06:08,251] INFO - jetbrains.buildServer.STARTUP - Current stage: TeamCity server is shutting down
[2018-12-07 16:06:08,251] INFO - jetbrains.buildServer.STARTUP - Stopping the main application
Please sign in to leave a comment.
I have just seen the following: https://youtrack.jetbrains.com/issue/TW-58289#
I will continue to investigate.
Update:
It appears I have about 10 files which look corrupted or encoded in the customDataStorage/buildTypes folder.
Example:
r$wpr¢xpo0 +oJ
*Zo5
r$wpr¢xpo0 +*j,o5
r$wpr¢xpo1 +*(8
*{Z *Æ{Z (3
,"r¶spo0
(4
Are these okay to remove?
Hi Ryan,
sorry for the inconveniences.Your issue is the one you referenced, and the workarounds and core issues are described on the tracker you linked. There is 2 jars that should help with the issue going forward (while we fix it for the next bugfix release), and in the meantime, removing the corrupted files is not just safe, but pretty much mandatory. Please check the public comments on the issue tracker, if you face any further issue with the upgrade process, please comment in the issue on the tracker instead of here, as that way your messages will be directly received by the responsible devs.