A dozen "BuildServer" directories ... which to back up??

When I installed TeamCity, I configured against a MySQL database, allowed the TeamCity data directory to be the default and configured TeamCity to run as a Windows Service as a set user (server and agent both run as "Local System"). It has been running pretty uninterrupted (with the occasional system restart) on Windows Server 2003 for just over a year now, accumulating approximately 8,000 run builds. Today we ran out of disk space (100GB hard disk) and I realized no cleanup policy had ever been configured. So I configured one and ran the cleanup, supposedly cleaning up the logs for some 7,600 builds and clearing up 85GB of disk space.

Instead, only 22GB of disk space were cleared out. Confused, I ran the free TreeSize program and discovered that there were TWELVE "BuildServer" directories. One at C:\BuildServer and the others: one in each user's directory in Documents and Settings. The one at C:\BuildServer is my installation directory, with a subdirectory TeamCity underneath it and, underneath that, temp, buildAgent, webapps, jre, logs, work, lib, devPackage, bin, license and conf.

Under each user directory in "Documents and Settings" is another directory called "BuildServer," and under that is "system", "config" and "plugins" and under "system" is "messages", "caches" and "artifacts". The "messages" directories under five of the users are HUGE ... one is 25GB, one is 15GB, one is 11GB, one is 10GB and the other is 7GB (all these sizes AFTER the cleanup). The other users all have relatively small "messages" directories, but they ARE still there for EVERY user (including the default system users).

My first question is, how in the world did this happen? How did my data directory get split across so many user accounts?

Now, I'm guessing I'm in a whole heap of trouble, because I need to be able to clean up this data. I'm also on version 5.0 and I'm thinking about upgrading to 6.x, so I know I'm going to need to back up my setup to upgrade. I don't mind if, just this once, I lose ALL of my history. That's not a problem for me. But...

My second questions is, what in the world do I back up (besides, obviously, my MySQL database)? I'm NOT okay with losing any of my build configurations or settings or users. I'm okay with losing my history, but that's it. So what do I backup?

Finally, I need to stop this from happening again. So...

My third question is, what recommendations do y'all have to keep such bizarre behavior from being repeated in the future?

Thanks in advance for the answers and advice,

Nick

4 comments
Comment actions Permalink

Hi Nicholas,

The .Buildserver directory that is really used by TeamCity server you can see on the web UI - go to the "Administration / Configuration" page. This page shows the real Data Directory path, type of database that is used by TeamCity, Authentication scheme, etc.

If you want to perform a backup - I'm recommending you to use backup from web UI (see http://confluence.jetbrains.net/display/TCD5/Backup+Data ) how to manage it. using this function you can get a backup file that contain configurations, users accounts, build history, build logs but no artifacts. Note that backup that is created by TeamCity 6.x can be restored only into the same version of TeamCity (since 6.0 that restriction is dropped). When backup is done you can save the backup file into another computer to prevent mixing it with other files in other directories...

> My first question is, how in the world did this happen? How did my data directory get split across so many user accounts?

The first option is if this server is an NT domain controller for your users and it is set up to copy their profiles to server (windows user profiles roaming, see http://en.wikipedia.org/wiki/Roaming_user_profile for details). In this case the .Buildserver directories don't relate to the TC server installed on this computer.

One more case how such directories can be created is if users started their own TC servers from their accounts.

It might be other ways how such directories can be created, I could not answer this question because I didn't see these ones.

> My second questions is, what in the world do I back up (besides, obviously, my MySQL database)?

See my recomendations above.
Additionally all backup options are described here: http://confluence.jetbrains.net/display/TCD5/TeamCity+Data+Backup

> My third question is, what recommendations do y'all have to keep such bizarre behavior from being repeated in the future?

The answer on this question depends on the first question; so in order to prevent new strange directories to be created you need to investigate who is creating them...


Best regards,
Leonid

0
Comment actions Permalink

Leonid,

Thanks so much for the helpful reply!

I was able to find the correct TeamCity data directory in the configuration just like you said. Now I know what to backup. :)

We do not use roaming user profiles, so I can only figure that the other users on the machine somehow, at one time or another, started TeamCity as them instead of as the default user. So the only thing I still don't understand is, since those other users accumulated tens of gigabytes of build logs (meaning at least hundreds of builds were run as them), how we never lost our build server configuration when the data directory changed due to the running user changing.

So, assuming I'm right and that's how the multiple directories were created, how can I prevent that from happening again? Can you configure a static data directory that is outside of and doesn't care about the user profile directory?

Thanks!

Nick

0
Comment actions Permalink

There is an article in our documentation.
In short, when the server is started as Windows service, it gets the path from Tomcat properties.
If it's started interactivelly by runAll.bat/teamcity-server.bat, then it checks %TEAMCITY_DATA_PATH% system variable, or looks for %USERPROFILE%\.buildServer, which is different for every account.
It's still unclear, how copies within your user profiles got populated by build history data. But if you set %TEAMCITY_DATA_PATH% as system-wide variable and point it to the same location, as specified in Tomcat settings, it'll guarantee the server uses the same configuration no matter how it's started.

Michael

0
Comment actions Permalink

Excellent! When I upgrade to TC6 next week, I will be sure to set that setting in both places to a static directory outside of the user profile directories. Hopefully I won't have to deal with this issue anymore after that.

Thanks for all of the help, guys!

0

Please sign in to leave a comment.