After upgrade to 8.0.4, half my projects have disappeared. Help.

I upgraded to 8.0.4 from 8.0.3.  We had two sub projects under root, each of which had additional subprojects under that.  One of those subprojects is completely gone after the upgrade.

There were no errors or messages in the logs during the upgrade that indicated a problem, or that these projects would be deleted.

The UI has virtually no indication that the projects ever existed.  No audit of them being removed.  They aren't in the 'archived' project list.  VCS roots that were associated with those projects are now associated with the root project.  audit logs of past builds in those projects just list the internal ID of the old build configurations, and clicking on those IDs indicates those build configurations no longer exist.

I looked in the database, in the project table.  The top-level subproject *is* listed there, but not the subprojects under that.  Not sure where else to look.

I tried downgrading back to 8.0.3, but that didn't bring back the missing projects.

I unfortunately don't have a recent backup to restore from (I know...).  I was hoping you could point me at database tables or conf files where some of this configuration might still be so I could perhaps manually try and undo this.

I'm running the war distribution in tomcat7 on ubuntu 12.

Comment actions Permalink

I think I've recovered.  I logged into the teamcity database, listed the contents of the 'project' table, identified the row of the missing sub project, went into the UI and created a new subproject with that same id.

I then went back to the main page and expanded that new subproject, and all my build configurations were back.

I thought I had more subprojects under there, but I now think I didn't.  It was just that one missing subproject.

Whew.  That was very scary.

Comment actions Permalink

What was the ID of this subproject? Were there any messages in teamcity-server.log about it?

Comment actions Permalink

The internal id was SafeMonkClient.

I don't see any messages in the teamcity-server logs that seem related.  There are a bunch of 'Cleaning artifacts' messages...grep those out, and there are some other cleaning related messages, some messages about builds being queued (after I recovered from the missing projects).  

There are some error messages that look like this:

Failed to write to file: {path to build folder}/system/artifacts/SafeMonkClient/OSX Client Incremental/8692/.teamcity/settings/digest.txt, error: {path to build folder}/system/artifacts/SafeMonkClient/OSX Client Incremental/8692/.teamcity/settings/digest.txt (No such file or directory)

Unable to read properties from {path to build folder}/config/projects/SafeMonkClient/buildNumbers/ 0

WARN - rverSide.impl.BuildNumbersImpl - Cannot read numeric build counter from file {path to build folder}/config/projects/SafeMonkClient/buildNumbers/ Will reset value to 1.

But these errors were all in a period 4 days before the upgrade, when we had run out of disk space and had to restart the server, so I don't think they are related.  There are no error messages or warnings around the time of the upgrade.


Please sign in to leave a comment.