TeamCity restore from backup: "Critical error in configuration file" and "Failed to find parent project with ID" Follow
Hello,
We performed a restore from backup procedure and everything worked OK.
Our restore command was:
~# ./maintainDB.sh restore -A /data/teamcity-data/ -F /data/backup/TeamCity_Backup_20200222_181005.zip -T /data/backup/database.properties
Now the /data/teamcity-data folder contains all the files restored. I also copied the database.properties to the /data/teamcity-data/config/ folder.
Now, when we started TeamCity we get the following errors (continuously, for all projects) in the logs/teamcity-server.log file:
========== BEGIN LOGS SNIPPET ==========
[2020-02-25 17:38:45,744] WARN - jetbrains.buildServer.SERVER - Found critical error in configuration: Critical error in configuration file /data/teamcity-data/config/projects/PlatformConsole_LoomPlugins_UxPlu/project-config.xml:
Failed to find parent project with ID "PlatformConsole_LoomPlugins" for project with ID "PlatformConsole_LoomPlugins_UxPlu"
[2020-02-25 17:38:45,744] WARN - jetbrains.buildServer.SERVER - Critical error in configuration file: /data/teamcity-data/config/projects/PlatformConsole_LoomPlugins_UxPlu/project-config.xml, error: jetbrains.buildServer.serverSide.impl.UnresolvedParentProjectException: Failed to find parent project with ID "PlatformConsole_LoomPlugins" for project with ID "PlatformConsole_LoomPlugins_UxPlu": jetbrains.buildServer.serverSide.impl.UnresolvedParentProjectException: Failed to find parent project with ID "PlatformConsole_LoomPlugins" for project with ID "PlatformConsole_LoomPlugins_UxPlu"
[2020-02-25 17:38:45,744] DEBUG - jetbrains.buildServer.SERVER - Critical error in configuration file: /data/teamcity-data/config/projects/PlatformConsole_LoomPlugins_UxPlu/project-config.xml, error: jetbrains.buildServer.serverSide.impl.UnresolvedParentProjectException: Failed to find parent project with ID "PlatformConsole_LoomPlugins" for project with ID "PlatformConsole_LoomPlugins_UxPlu"
jetbrains.buildServer.serverSide.impl.UnresolvedParentProjectException: Failed to find parent project with ID "PlatformConsole_LoomPlugins" for project with ID "PlatformConsole_LoomPlugins_UxPlu"
at jetbrains.buildServer.serverSide.impl.projects.ProjectDataModelImpl$InnerModel$ProjectConstraints.afterObjectLoad(ProjectDataModelImpl.java:2)
at jetbrains.buildServer.serverSide.impl.projects.ProjectDataModelImpl$InnerModel$ProjectConstraints.afterObjectLoad(ProjectDataModelImpl.java:30)
at jetbrains.buildServer.serverSide.impl.IndexedCollection.fireAfterObjectLoad(IndexedCollection.java:50)
at jetbrains.buildServer.serverSide.impl.IndexedCollection.afterObjectLoad(IndexedCollection.java:111)
at jetbrains.buildServer.serverSide.impl.projects.ProjectDataModelImpl$InnerModel.afterProjectLoad(ProjectDataModelImpl.java:78)
at jetbrains.buildServer.serverSide.impl.projects.ProjectDataModelImpl$InnerModel.access$800(ProjectDataModelImpl.java:108)
at jetbrains.buildServer.serverSide.impl.projects.ProjectDataModelImpl.afterProjectLoad(ProjectDataModelImpl.java:121)
at jetbrains.buildServer.serverSide.impl.projects.ProjectModelWrapper.afterProjectLoad(ProjectModelWrapper.java:325)
at jetbrains.buildServer.serverSide.impl.projects.ProjectsLoader.processChangedConfigs(ProjectsLoader.java:303)
at jetbrains.buildServer.serverSide.impl.projects.ProjectsLoader.reloadProjects(ProjectsLoader.java:432)
at jetbrains.buildServer.serverSide.impl.projects.ProjectsLoader.loadAllProjects(ProjectsLoader.java:656)
at jetbrains.buildServer.serverSide.impl.projects.ProjectsWatcher.loadProjects(ProjectsWatcher.java:65)
at jetbrains.buildServer.serverSide.impl.xml.ConfigurationInitializer.loadConfiguration(ConfigurationInitializer.java:17)
at jetbrains.buildServer.serverSide.impl.xml.ConfigurationInitializer.access$200(ConfigurationInitializer.java:7)
at jetbrains.buildServer.serverSide.impl.xml.ConfigurationInitializer$1.run(ConfigurationInitializer.java:2)
at jetbrains.buildServer.serverSide.impl.cleanup.ServerCleanupState.runUnderLock(ServerCleanupState.java:10)
at jetbrains.buildServer.serverSide.impl.cleanup.ServerCleanupState.runReadAction(ServerCleanupState.java:83)
at jetbrains.buildServer.serverSide.impl.cleanup.ServerCleanupManagerImpl.executeWithInactiveCleanup(ServerCleanupManagerImpl.java:207)
at jetbrains.buildServer.serverSide.impl.cleanup.ServerCleanupManagerImpl$$FastClassBySpringCGLIB$$ba2c8525.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at jetbrains.buildServer.serverSide.impl.auth.TeamCityMethodSecurityInterceptor.invoke(TeamCityMethodSecurityInterceptor.java:28)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.CglibAopProxy$FixedChainStaticTargetInterceptor.intercept(CglibAopProxy.java:622)
at jetbrains.buildServer.serverSide.impl.cleanup.ServerCleanupManagerImpl$$EnhancerBySpringCGLIB$$1f86ddfb.executeWithInactiveCleanup(<generated>)
at jetbrains.buildServer.serverSide.impl.xml.ConfigurationInitializer.initServer(ConfigurationInitializer.java:2)
at jetbrains.buildServer.web.impl.BuildServerConfigurator.loadConfiguration(BuildServerConfigurator.java:7)
at java.util.concurrent.CopyOnWriteArrayList.forEach(CopyOnWriteArrayList.java:891)
at jetbrains.buildServer.serverSide.impl.BuildServerLifecycleProcessor.doStartup(BuildServerLifecycleProcessor.java:26)
at jetbrains.buildServer.maintenance.TeamCityDispatcherServlet$WebApplicationCreatorAndDestroyer.createApplication(TeamCityDispatcherServlet.java:30)
at jetbrains.buildServer.maintenance.StartupProcessor.doApplicationStarting(StartupProcessor.java:1033)
at jetbrains.buildServer.maintenance.StartupProcessor.access$1500(StartupProcessor.java:79)
at jetbrains.buildServer.maintenance.StartupProcessor$3.call(StartupProcessor.java:10)
at jetbrains.buildServer.util.NamedThreadFactory.executeWithNewThreadName(NamedThreadFactory.java:91)
at jetbrains.buildServer.maintenance.StartupProcessor.processConcreteStage(StartupProcessor.java:1056)
at jetbrains.buildServer.maintenance.StartupProcessor.processConcreteStageSafe(StartupProcessor.java:16)
at jetbrains.buildServer.maintenance.StartupProcessor.processTeamCityLifecycle(StartupProcessor.java:883)
at jetbrains.buildServer.maintenance.StartupProcessor.access$000(StartupProcessor.java:252)
at jetbrains.buildServer.maintenance.StartupProcessor$1.run(StartupProcessor.java:1)
at java.lang.Thread.run(Thread.java:748)
========== END ==========
But, the /data/teamcity-data/config/projects/PlatformConsole_LoomPlugins/ folder exists (and it contains the the project-config.xml file).
And all the file permissions are set correct.
And, if I follow the /data/teamcity-data/config/projects/PlatformConsole_LoomPlugins/project-config.xml and check for its parent-id (which is PlatformConsole) and then keep going by looking at /data/teamcity-data/config/projects/PlatformConsole/project-config.xml until I find no more "parent-id" defined, then chain of dependencies looks OK and all the folders and files exists in the right place with the right ownership and permissions.
This issue may be related to https://youtrack.jetbrains.com/issue/TW-28886, but I did all the lookup of the dependencies and they all are OK.
Do you know why we get these errors? Is there anything else I can check in order to figure out why TeamCity complains about "Failed to find parent with object ID"?
Thank you,
George.
Please sign in to leave a comment.
Hello George,
In the case described in TW-28886, the errors are displayed because the project hierarchy cannot be resolved (due to a missing top-level project which was not copied over).
To verify what projects are present right now in the server's memory you can execute a REST API request like this (using a system administrator account):
http(s)://teamcity-server/app/rest/projects?fields=$long,project(id,internalId,name,parentProjectId)
That will show you the list of the projects with their IDs and their parent project IDs. If the situation is different from what you are seeing in the data directory, a server restart may help to actualize it.
-Anatoly
Hello Anatoly,
Executing the REST API on the original instance won't help - the main instance starts/stop (teamcity-server.sh start or stop) successfully without printing any of those errors.
The issue happens when, using a backup (taken using maintainDB.sh), we try to restore it on a different system.
But, after we did the restore (which takes about 9 hours, we have a lot of data) and start TeamCity - and observe the errors in the logs - then we stop TeamCity, and start it again, then on the second start it does not logs those errors anymore. It is strange it only logs the errors on the first startup.
I will repeat the procedure a few times, with newer backups, over the next week and provide the findings here.
Thank you,
George.