TeamCity thinks multiple servers are connecting to the database

We are running TeamCity 5.0.1 and have just migrated from 4.5.5.  When we start up TeamCity and hit the servlet we are now seeing:

jetbrains.buildServer.rootDispatcher.SecondTeamCityInstanceException: Attempted to start more than one TeamCity servers connected with one database.
   at jetbrains.buildServer.rootDispatcher.TeamCityDispatcherServlet.processStartupServerInternal(TeamCityDispatcherServlet.java:101)
   at jetbrains.buildServer.rootDispatcher.TeamCityDispatcherServlet.processServerStartup(TeamCityDispatcherServlet.java:46)
   at jetbrains.buildServer.rootDispatcher.TeamCityDispatcherServlet.access$000(TeamCityDispatcherServlet.java:77)
   at jetbrains.buildServer.rootDispatcher.TeamCityDispatcherServlet$1.run(TeamCityDispatcherServlet.java:2)
   at java.lang.Thread.run(Thread.java:619)


This database was previously used in TeamCity 4.5.5 and was working for a few minutes (before we rebooted the OS -- Linux).   We created an empty database which does work, but we'd really like to keep the old data!

We are really sure that only one instance of TeamCity is running against this database (a MySQL database).  Is there some lock file or table that we can clear?

Thanks,
DBrownell
15 comments
Comment actions Permalink

David,

You can try to manually kill all connections from the user that TeamCity is using to connect to your database (your DBA should know how) and then restart TeamCity. Sometimes those connections are still there after you restart the server, so it doesn't let you connect what the DB thinks is another TC instance. Hope this helps!

0
Comment actions Permalink

Have you tried to take a look at connections opened to MySQL server? I mean the command: show processlist
Does the list contains connections from the one server only?

0
Comment actions Permalink

What we ended up doing was to create a new database and dump and load the user tables into the new DB.  Sure, we lost our history and if having the history was more important we'd persue getting that over.  I was hoping that when a server connected, it threw a flag that we might clear.  The only thing connecting to the database was our TeamCity server, and we ensured that we shut down the old 4.x server before cranking up the new one.

Thanks for the suggestions.

0
Comment actions Permalink

In fact TeamCity uses get_lock MySQL function (http://dev.mysql.com/doc/refman/5.1/en/miscellaneous-functions.html#function_get-lock) to obtain lock on the TeamCity database, and if it fails to do so, the error you've seen appears. We need to lock database because in the past we had a lot of issues when more than one server pointed to the same database. The problem is in Tomcat it is very easy to make a mistake in configuration which leads to such case. Many users did not know they run more than one instance of TeamCity until some primary key violation occurred.

It would be great if we could investigate why the server detection approach does not work in your case. What version of MySQL/JDBC driver do you use?

0
Comment actions Permalink

I'm getting the same thing when I go to our TeamCity webpage.  We are running our database on SQL Server 2005. Any ideas if this is a TeamCity defect causing this or if we really have another instance trying to use the same database, which would be unlikly. We are on TeamCity 5.0.1.

Ooops, something went wrong...

Error on step: Connecting to the database server...

Attempted to start more than one TeamCity servers connected with one  database.


Copy to clipboard

Stacktrace copied!

jetbrains.buildServer.rootDispatcher.SecondTeamCityInstanceException:  Attempted to start more than one TeamCity servers connected with one  database.

   at  jetbrains.buildServer.rootDispatcher.TeamCityDispatcherServlet.processStartupServerInternal(TeamCityDispatcherServlet.java:134)

   at  jetbrains.buildServer.rootDispatcher.TeamCityDispatcherServlet.processServerStartup(TeamCityDispatcherServlet.java:46)

   at  jetbrains.buildServer.rootDispatcher.TeamCityDispatcherServlet.access$000(TeamCityDispatcherServlet.java:34)

   at  jetbrains.buildServer.rootDispatcher.TeamCityDispatcherServlet$1.run(TeamCityDispatcherServlet.java)

   at java.lang.Thread.run(Thread.java:595)


Bob

0
Comment actions Permalink

Guys,

could you please check how many TeamCity instances are really running?
It can be done, for instance, using Tomcat Manager (just browse the page http://<myserver.mydomain.com>/manager/html).

Thanks.

--
Leonid

0
Comment actions Permalink

When I do that all I get is this page.  Also to note we upgraded to 5.0.2 to see if our original issue would be fixed and we still get the same error. It seems the app and db version checking my be off our database is 345 but the log shows two app version one set to 344 and one to 345. This is logged as an error before the multiple errors issue is logged.

HTTP Status 404 - /manager/html/


type Status report
message /manager/html/
description The requested resource (/manager/html/) is not available.


Apache Tomcat/6.0.20

0
Comment actions Permalink

Bob,
this page is displayed when no Tomcat Mangers installed.

In this case, you can look thru log file "teamcity-server.log" that is placed in the <TeamCity-home>/logs directory to check how many instances are starting.

P.S. How did you install TeamCity - using Windows installer or using WAR-file or from tar.gz?

--
Leonid

0
Comment actions Permalink

Bob,

could you please provide me this log?

Thank you,

--
Leonid

0
Comment actions Permalink

We installed 5.0.1 with windows installer. We ran into MSSQL blocking bug. Got a pre 5.0.2 fix number 10752 and installed the war to get around MSSQL bug. Then we ran the official 5.0.2 installer to get that up to offical 5.0.2. Which now we have the multiple server connection issue.

0
Comment actions Permalink

I sent you an email.

0
Comment actions Permalink

Bob,

it looks like you have two versions of TeamCity both rinning under one Tomcat server. Please, remove the old one (completely, not just rename or zip). I hope it will help.

Please, describe how did you install/upgrade TeamCity - it will help us to undestand the case.

Thank you,

Leonid

0
Comment actions Permalink

Bob,

just remove the old version completely (not rename or zip) and start the server.

Best regards,

Leonid

0
Comment actions Permalink

We installed 5.0.1 with windows installer. We ran into MSSQL blocking bug. Got a pre 5.0.2 fix number 10752 and installed the war to get around MSSQL bug. Then we ran the official 5.0.2 installer to get that up to offical 5.0.2. Which now we have the multiple server connection issue.



We fixed our issue by having a backup of .BuildServer directory. Doing a uninstall via windows add/remove gui. We then installed fresh 5.0.2 and copied back our backed up .BuildServer.  Coping back our jtds jar file for the MSSQL database connection.

Thanks for your help into looking into this.  It appears that pre release war we manually intalled was most likely to blame.

0
Comment actions Permalink

I just got this error with version 5.1.4 on ms sql 2008, I made a backup of the teamcity database, and restored it again and that fixed the issue for me.

0

Please sign in to leave a comment.