Problem migrating from one Oracle DB to another using maintainDB

Hi,

I just migrated my TeamCity database from one schema to another. No errors were reported by maintainDB and in fact all data appears to have been successfully imported. However, no when I start TeamCity, I get the following error:

jetbrains.buildServer.serverSide.db.SecondTeamCityInstanceException: Attempt to start more than one TeamCity server connected to the same database.
   at jetbrains.buildServer.serverSide.db.TeamCityDatabaseManager.lock(TeamCityDatabaseManager.java:91)
   at jetbrains.buildServer.rootDispatcher.TeamCityDispatcherServlet.processStartupServerInternal(TeamCityDispatcherServlet.java:80)
   at jetbrains.buildServer.rootDispatcher.TeamCityDispatcherServlet.processServerStartup(TeamCityDispatcherServlet.java:60)
   at jetbrains.buildServer.rootDispatcher.TeamCityDispatcherServlet.access$000(TeamCityDispatcherServlet.java:76)
   at jetbrains.buildServer.rootDispatcher.TeamCityDispatcherServlet$1.run(TeamCityDispatcherServlet.java:2)
   at java.lang.Thread.run(Unknown Source)



Here's the maintainDB command line I used... the only potential gotcha I can think of is the source db was running TC5.0 while the target was running 5.1.

D:\TeamCity5.1\bin>maintainDB migrate -A D:\TeamCity5.1\.BuildServer -S D:\TeamCity5.1\.BuildServer\config\migrate\database.oracle.source.properties -T D:\TeamCity5.1\.BuildServer\config\migrate\database.oracle.properties

Anyone seen this before or have any ideas how to resolve? Is it as simple as finding some row in one of the tables and deleting it?

Dan
16 comments

Hello Dan,

do you start both TC servers simultaneously? if yes, could you please provide me with two files 'database.properties' (for one from each installation).
If no (you're starting just one TC installation), could you please double check that TC is not already running (and is stopped completely) when you are starting TC.

Best reagrds,
Leonid

0

Leonid,

Thanks for the quick response. I'm actually resetting the test now with both app servers down. I'm going to have my dba drop the schema for the 5.1 instance and I'll rerun the maintainDB command again. Let's see how it goes.

Regards,

Dan

0

Unforunately, even after repeating the process from scratch I'm getting the same result. These are the steps I took:

  1. took down both TeamCity instances ( each running on different boxes ) and changed the services to restart manually
  2. restarted each appserver host ensuring that neither had any TeamCity services running after reboot
  3. drop all objects in target oracle schema
  4. reran maintainDB ( see attached output and properties files )
  5. verified database.properties in $DATADIR\config had the connection string from the target schema ( teamcity2 )
  6. restarted TeamCity on the target machine - same exception


jetbrains.buildServer.serverSide.db.SecondTeamCityInstanceException: Attempt to start more than one TeamCity server connected to the same database.
   at jetbrains.buildServer.serverSide.db.TeamCityDatabaseManager.lock(TeamCityDatabaseManager.java:91)
   at jetbrains.buildServer.rootDispatcher.TeamCityDispatcherServlet.processStartupServerInternal(TeamCityDispatcherServlet.java:80)
   at jetbrains.buildServer.rootDispatcher.TeamCityDispatcherServlet.processServerStartup(TeamCityDispatcherServlet.java:60)
   at jetbrains.buildServer.rootDispatcher.TeamCityDispatcherServlet.access$000(TeamCityDispatcherServlet.java:76)
   at jetbrains.buildServer.rootDispatcher.TeamCityDispatcherServlet$1.run(TeamCityDispatcherServlet.java:2)
   at java.lang.Thread.run(Unknown Source)


maintainDB CommandLine:
./maintainDB migrate -A D:\TeamCity5.1\.BuildServer -S D:\TeamCity5.1\.BuildServer\config\migrate\database.oracle.source.properties -T D:\TeamCity5.1\.BuildServer\config\migrate\database.oracle.target.properties

Contents of database.oracle.source.properties
connectionUrl=jdbc:oracle:thin:@TECHSTAR6.techlab.local:1521:TS6CRMS
connectionProperties.user=teamcity
connectionProperties.password=teamcity10


Contents of database.oracle.target.properties
connectionUrl=jdbc:oracle:thin:@TECHSTAR6.techlab.local:1521:TS6CRMS
connectionProperties.user=teamcity2
connectionProperties.password=teamcity2


Attachment(s):
maintainDB.out.zip
0

Dan,

all you have done look right. Let me try to reproduce this on my side. I'll inform you with my result.

Thanks,
Leonid

0

Dan,

one moment also - are you starting TeamCity as a Windows service? if yes, could you please double check the 'teamcity-server.log' file that it really uses the NEW data directory on the target box.

Thanks,
Leonid

0

Leonid,

I checked the teamcity-server.log and it is pointing to the correct data directory. Here is the output:

[2010-07-08 13:34:57,856]   INFO -   jetbrains.buildServer.SERVER - Starting TeamCity server
[2010-07-08 13:34:58,387]   INFO - dServer.serverSide.ServerPaths - TeamCity data directory: d:/TeamCity5.1/.BuildServer/
[2010-07-08 13:34:58,966]   INFO -   jetbrains.buildServer.SERVER - TeamCity configuration version is 378
[2010-07-08 13:35:20,591]  ERROR -   jetbrains.buildServer.SERVER - Server startup exception: jetbrains.buildServer.serverSide.db.SecondTeamCityInstanceException: Attempt to start more than one TeamCity server connected to the same database.



The line with configuration version as 378... is that significant? Does TeamCity store state in the database? Meaning maybe this is just a question of the old server having written one value and that getting carried over as part of the migration. Now that configuration version doesn't match the new TeamCity server. Just speculating here but is that possible?

Regards,
Dan
0

Hi Dan,

did you perform migration (starting maintainDB) on the original box or on the second box?

--
Leonid

0

Leonid,

I ran the maintainDB from the second box - the one I'm trying to migrate to.

Regards,
Dan

0

Dan,

could you please check the 'database.properties' file on the original box - whether it really points to the fisrt database?

Thanks,
Leonid

0

Leonid,

Sorry, been on vacation. Yes, the database.properties file on the first box points to the old schema as expected:

connectionUrl=jdbc:oracle:thin:@TECHSTAR6.techlab.local:1521:TS6CRMS
connectionProperties.user=teamcity
connectionProperties.password=teamcity10
connectionProperties.driver=oracle.jdbc.driver.OracleDriver



The database.properties file on the new box points to the new schema, also as expected:

connectionUrl=jdbc:oracle:thin:@TECHSTAR6.techlab.local:1521:TS6CRMS
connectionProperties.user=teamcity2
connectionProperties.password=teamcity2
connectionProperties.driver=oracle.jdbc.driver.OracleDriver



Dan
0

Sorry for delay,
I'll answer in a couple of days.

--
Leonid

0

Hi Dan,

could you please perform the following query (being logged in to Oracle as the SYS user with DBA privileges)

select *

from ( select name, sid, type, lmode, id1

         from v$lock L, dbms_lock_allocated A

         where type = 'UL'

           and L.id1 = A.lockid (+) )

     natural join

     ( select sid, audsid,

              username, schemaname, osuser, machine,

              lockwait, status, logon_time

         from v$session )


and provide me with results?

This query shows user-allocated locks that are captured by different users.

P.S. please perfrom it when the old server is started.

Thank you

--
Leonid

0

Leonid,

Here's the output:

SID NAME TYPE LMODE ID1 AUDSID USERNAME SCHEMANAME OSUSER MACHINE LOCKWAIT STATUS LOGON_TIME
46 TEAMCITY$TEAMCITY UL 6 1073741864 4261334 TEAMCITY TEAMCITY SYSTEM nrwpap06 [NULL] INACTIVE 7/19/2010 9:54:31.000 PM
48 TEAMCITY2$TEAMCITY UL 6 1073741865 4263051 TEAMCITY2 TEAMCITY2 [NULL] NEWDAP25 [NULL] INACTIVE 7/23/2010 1:12:09.000 PM


Regards,
Dan

 

0

Hi Dan,

as we can see two TeamCity servers currently work with this Oracle server.

First: user TEAMCITY from host nrwpap06 since 7/19/2010 9:54 pm.
Second: user TEAMCITY2 from host NEWDAP25 since 7/23/2010 1:12 pm.

Are this servers — two ones you wanted to run? Or one of them is "alien"?

Best regards,
Leonid

0

Leonid,

teamcity is the instance I'm trying to migrate from. This is the instance that is running 5.0 and does not have an enterprise license.

teamcity2 is the new instance running 5.1 that I'm trying to set up. It does have an unlimited enterprise license so the goal is to stop running the teamcity instance and replace with the teamcity2 instance. However, I cant' do that until I can migrate the data.

I would appreciate if this issue could be escalated as we've now been going in circles for over a week and not made any progress.

Regards,
Dan

0

Hi Dan,

I think you might misunderstood me — I'm not talking about licenses, just the technical side.

The log you've attached contains two records:

    1. user TEAMCITY from host nrwpap06
    2. user TEAMCITY2 from host NEWDAP25


Am I right that you performed this query when one (original) server was running but the target server was not running?
However the result contains two records — one for the original server (nrwpap06) and one for another one — NEWDAP25, that is not original and is not target. So there is a third TeamCity installation that puts obstacles in the way to start the target one.

Am I right?

Best regards,
Leonid
0

Please sign in to leave a comment.