Help! TeamCity is not starting up after migrate from HSQLDB to MSSQL 2005

Hi there,

I just have my TeamCity upgraded from HSQLDB to MSSQL, the Migration seems successful as the dbmigrate.bat does create a version.dat as the instruction told and I have got all the tables in MSSQL.
TeamCity service is able to start up, however, the page shows a 404 error:

HTTP Status 404 -


type Status report
message
description The requested resource () is not available.


Apache Tomcat/6.0.18


And I have checked the version.dat file created by the migration tool, it looks a bit strange to me:

     # generated by TC database migration tool @Thu Dec 04 16:48:51 EST 2008
     version.com.mysql.jdbc.Driver=266  

Where I migrating the DB to MS SQL but not MY SQL

I am pretty sure I have correctly edited the dbMigration.properties before running the dbmigrate.bat, the following is the modified parts:

This is the dbMigrate.properties

# Internal TeamCity database
sourceDriver=org.hsqldb.jdbcDriver
sourceURL=jdbc:hsqldb:file:D:/TeamCity/.BuildServer/system/buildserver
sourceUser=sa
sourcePassword=''

# MSSQL 2005 via jTDS
#targetDriver=net.sourceforge.jtds.jdbc.Driver
targetDriver=net.sourceforge.jtds.jdbc.Driver
targetURL=jdbc:jtds:sqlserver://W2DSQL01:4465/TeamCity
targetUser=login
targetPassword=password

This is the database.properties file:

# generated by TC database migration tool @Thu Dec 04 16:49:39 EST 2008
driverName=net.sourceforge.jtds.jdbc.Driver
connectionUrl=jdbc:jtds:sqlserver://W2DSQL01:4465/TeamCity
connectionProperties.user=login
connectionProperties.password=password

maxConnections=50
poolPreparedStatements=true

From the log files, I reckon the problem is a classpath problem but I have not touch anything that will change the path during the migration.

catalina.2008-12-04.log

4/12/2008 17:14:53 org.apache.catalina.core.AprLifecycleListener init
INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: D:\TeamCity\bin;.;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program Files\TortoiseSVN\bin;C:\ANT;C:\ANT\BIN;C:\Program Files\JAVA\jdk1.5.0_10\bin;d:\Build_Tools\nant
4/12/2008 17:14:53 org.apache.coyote.http11.Http11Protocol init
INFO: Initializing Coyote HTTP/1.1 on http-8000
4/12/2008 17:14:53 org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 730 ms
4/12/2008 17:14:53 org.apache.catalina.core.StandardService start
INFO: Starting service Catalina
4/12/2008 17:14:53 org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/6.0.18
4/12/2008 17:14:55 org.apache.catalina.core.StandardContext start
SEVERE: Error listenerStart
4/12/2008 17:14:55 org.apache.catalina.core.StandardContext start
SEVERE: Context [] startup failed due to previous errors
4/12/2008 17:14:55 org.apache.coyote.http11.Http11Protocol start
INFO: Starting Coyote HTTP/1.1 on http-8000
4/12/2008 17:14:55 org.apache.jk.common.ChannelSocket init
INFO: JK: ajp13 listening on /0.0.0.0:8009
4/12/2008 17:14:55 org.apache.jk.server.JkMain start
INFO: Jk running ID=0 time=0/31  config=null
4/12/2008 17:14:55 org.apache.catalina.startup.Catalina start
INFO: Server startup in 1947 ms

localhost.2008-12-04.log

4/12/2008 17:14:55 org.apache.catalina.core.ApplicationContext log
INFO: Initializing Spring root WebApplicationContext
4/12/2008 17:14:55 org.apache.catalina.core.StandardContext listenerStart
SEVERE: Exception sending context initialized event to listener instance of class org.springframework.web.context.ContextLoaderListener
org.springframework.beans.factory.BeanDefinitionStoreException: Parser configuration exception parsing XML from URL [jar:file:/D:/TeamCity/webapps/ROOT/WEB-INF/lib/server.jar!/META-INF/buildServerSpring.xml]; nested exception is javax.xml.parsers.ParserConfigurationException: Unable to validate using XSD: Your JAXP provider [org.apache.crimson.jaxp.DocumentBuilderFactoryImpl@402af3] does not support XML Schema. Are you running on Java 1.4 with Apache Crimson? Upgrade to Apache Xerces (or Java 1.5) for full XSD support.
    at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:412)
    at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:342)
    at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:310)
    at org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:143)
    at org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:178)
    at org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:149)
    at org.springframework.web.context.support.XmlWebApplicationContext.loadBeanDefinitions(XmlWebApplicationContext.java:124)
    at org.springframework.web.context.support.XmlWebApplicationContext.loadBeanDefinitions(XmlWebApplicationContext.java:92)
    at org.springframework.context.support.AbstractRefreshableApplicationContext.refreshBeanFactory(AbstractRefreshableApplicationContext.java:123)
    at org.springframework.context.support.AbstractApplicationContext.obtainFreshBeanFactory(AbstractApplicationContext.java:423)
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:353)
    at org.springframework.web.context.ContextLoader.createWebApplicationContext(ContextLoader.java:255)
    at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:199)
    at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:45)
    at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:3843)
    at org.apache.catalina.core.StandardContext.start(StandardContext.java:4342)
    at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
    at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
    at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:525)
    at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:926)
    at org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:889)
    at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:492)
    at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1149)
    at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:311)
    at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:117)
    at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
    at org.apache.catalina.core.StandardHost.start(StandardHost.java:719)
    at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
    at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
    at org.apache.catalina.core.StandardService.start(StandardService.java:516)
    at org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
    at org.apache.catalina.startup.Catalina.start(Catalina.java:578)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:585)
    at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:288)
    at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:413)
Caused by: javax.xml.parsers.ParserConfigurationException: Unable to validate using XSD: Your JAXP provider [org.apache.crimson.jaxp.DocumentBuilderFactoryImpl@402af3] does not support XML Schema. Are you running on Java 1.4 with Apache Crimson? Upgrade to Apache Xerces (or Java 1.5) for full XSD support.
    at org.springframework.beans.factory.xml.DefaultDocumentLoader.createDocumentBuilderFactory(DefaultDocumentLoader.java:102)
    at org.springframework.beans.factory.xml.DefaultDocumentLoader.loadDocument(DefaultDocumentLoader.java:70)
    at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:396)
    ... 37 more
Caused by: java.lang.IllegalArgumentException: No attributes are implemented
    at org.apache.crimson.jaxp.DocumentBuilderFactoryImpl.setAttribute(DocumentBuilderFactoryImpl.java:93)
    at org.springframework.beans.factory.xml.DefaultDocumentLoader.createDocumentBuilderFactory(DefaultDocumentLoader.java:99)
    ... 39 more
4/12/2008 17:14:55 org.apache.catalina.core.ApplicationContext log
INFO: Closing Spring root WebApplicationContext

stdout_20081204.log

[2008-12-04 17:14:55,628]  ERROR - work.web.context.ContextLoader - Context initialization failed
org.springframework.beans.factory.BeanDefinitionStoreException: Parser configuration exception parsing XML from URL [jar:file:/D:/TeamCity/webapps/ROOT/WEB-INF/lib/server.jar!/META-INF/buildServerSpring.xml]; nested exception is javax.xml.parsers.ParserConfigurationException: Unable to validate using XSD: Your JAXP provider [org.apache.crimson.jaxp.DocumentBuilderFactoryImpl@402af3] does not support XML Schema. Are you running on Java 1.4 with Apache Crimson? Upgrade to Apache Xerces (or Java 1.5) for full XSD support.
    at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:412)
    at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:342)
    at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:310)
    at org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:143)
    at org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:178)
    at org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:149)
    at org.springframework.web.context.support.XmlWebApplicationContext.loadBeanDefinitions(XmlWebApplicationContext.java:124)
    at org.springframework.web.context.support.XmlWebApplicationContext.loadBeanDefinitions(XmlWebApplicationContext.java:92)
    at org.springframework.context.support.AbstractRefreshableApplicationContext.refreshBeanFactory(AbstractRefreshableApplicationContext.java:123)
    at org.springframework.context.support.AbstractApplicationContext.obtainFreshBeanFactory(AbstractApplicationContext.java:423)
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:353)
    at org.springframework.web.context.ContextLoader.createWebApplicationContext(ContextLoader.java:255)
    at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:199)
    at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:45)
    at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:3843)
    at org.apache.catalina.core.StandardContext.start(StandardContext.java:4342)
    at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
    at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
    at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:525)
    at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:926)
    at org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:889)
    at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:492)
    at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1149)
    at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:311)
    at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:117)
    at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
    at org.apache.catalina.core.StandardHost.start(StandardHost.java:719)
    at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
    at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
    at org.apache.catalina.core.StandardService.start(StandardService.java:516)
    at org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
    at org.apache.catalina.startup.Catalina.start(Catalina.java:578)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:585)
    at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:288)
    at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:413)
Caused by: javax.xml.parsers.ParserConfigurationException: Unable to validate using XSD: Your JAXP provider [org.apache.crimson.jaxp.DocumentBuilderFactoryImpl@402af3] does not support XML Schema. Are you running on Java 1.4 with Apache Crimson? Upgrade to Apache Xerces (or Java 1.5) for full XSD support.
    at org.springframework.beans.factory.xml.DefaultDocumentLoader.createDocumentBuilderFactory(DefaultDocumentLoader.java:102)
    at org.springframework.beans.factory.xml.DefaultDocumentLoader.loadDocument(DefaultDocumentLoader.java:70)
    at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:396)
    ... 37 more
Caused by: java.lang.IllegalArgumentException: No attributes are implemented
    at org.apache.crimson.jaxp.DocumentBuilderFactoryImpl.setAttribute(DocumentBuilderFactoryImpl.java:93)
    at org.springframework.beans.factory.xml.DefaultDocumentLoader.createDocumentBuilderFactory(DefaultDocumentLoader.java:99)
    ... 39 more

jakarta_service_20081204.log

[2008-12-04 17:14:52] [info] Procrun (2.0.4.0) started
[2008-12-04 17:14:52] [info] Running Service...
[2008-12-04 17:14:52] [info] Starting service...
[2008-12-04 17:14:54] [info] Service started in 1125 ms.

And there's nothing useful in the teamcity-server.log.

Anyone got an idea? This is killing me as this is a production server. >"<


BR
Thomas

9 comments

Hello,
   
I see a couple of strange things.   

First, your version.dat should contain a line 
   version.net.sourceforge.jtds.jdbc.Driver=266

I suppose that this is a bug in the DB migration tool. You can add this line to version.dat file.   


The exception in console is pretty strange. Are you sure you start TeamCity with JDK 1.5+?   


Regards,  
KIR

0

Thomas,

The root cause of the error seems to be:

Unable to validate using XSD: Your JAXP provider [org.apache.crimson.jaxp.DocumentBuilderFactoryImpl@402af3] does not support XML Schema. Are you running on Java 1.4 with Apache Crimson? Upgrade to Apache Xerces (or Java 1.5) for full XSD support.


Please ensure you run the server using Java 1.5.
Were there any other changes to your environment except for database migration?

--
Best regards,

Yegor Yarko
Project Manager (TeamCity)
JetBrains, Inc
http://www.jetbrains.com
"Develop with pleasure!"

0

The root cause of the error seems to be:


Unable to validate using XSD: Your JAXP provider [org.apache.crimson.jaxp.DocumentBuilderFactoryImpl@402af3] does not support XML Schema. Are you running on Java 1.4 with Apache Crimson? Upgrade to Apache Xerces (or Java 1.5) for full XSD support.



Please ensure you run the server using Java 1.5.
Were there any other changes to your environment except for database migration?

C:\Documents and Settings\tchau>set
ALLUSERSPROFILE=C:\Documents and Settings\All Users
ANT_HOME=D:\TeamCity\buildAgent\plugins\ant
APPDATA=C:\Documents and Settings\tchau\Application Data
CATALINA_HOME=D:\TeamCity
Classpath=D:\TeamCity\buildAgent\plugins\ant;D:\TeamCity\buildAgent\plugins\ant\bin;C:\Pro
gram Files\JAVA\jdk1.5.0_10\bin;C:\Program Files\JAVA\jdk1.5.0_10;D:\TeamCity\jre;C:\Progr
am Files\JAVA\jdk1.5.0_10;D:\TeamCity\jre\bin;D:\TeamCity\jre\lib
CLIENTNAME=LAPTOP80
ClusterLog=C:\WINDOWS\Cluster\cluster.log
CommonProgramFiles=C:\Program Files\Common Files
COMPUTERNAME=W2DCONFIG01
ComSpec=C:\WINDOWS\system32\cmd.exe
FP_NO_HOST_CHECK=NO
HOMEDRIVE=C:
HOMEPATH=\Documents and Settings\tchau
JAVA_HOME=C:\Program Files\JAVA\jdk1.5.0_10
LOGONSERVER=\\W2PAD01
NANTHOME=d:\Build_Tools\nant
NUMBER_OF_PROCESSORS=2
OS=Windows_NT
Path=C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program Files\TortoiseSVN\
bin;C:\ANT;C:\ANT\BIN;C:\Program Files\JAVA\jdk1.5.0_10\bin;d:\Build_Tools\nant
PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH
PROCESSOR_ARCHITECTURE=x86
PROCESSOR_IDENTIFIER=x86 Family 6 Model 23 Stepping 8, GenuineIntel
PROCESSOR_LEVEL=6
PROCESSOR_REVISION=1708
ProgramFiles=C:\Program Files
PROMPT=$P$G
SESSIONNAME=RDP-Tcp#1
SVN_ASP_DOT_NET_HACK=*
SystemDrive=C:
SystemRoot=C:\WINDOWS
TEAMCITY_SERVER_OPTS=-agentlib:yjpagent
TEMP=C:\DOCUME~1\tchau\LOCALS~1\Temp\1
TMP=C:\DOCUME~1\tchau\LOCALS~1\Temp\1
USERDNSDOMAIN=OASISCORP.COM.AU
USERDOMAIN=OASISCORP
USERNAME=tchau
USERPROFILE=C:\Documents and Settings\tchau
windir=C:\WINDOWS


C:\Documents and Settings\tchau>java -version
java version "1.5.0_10"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_10-b03)
Java HotSpot(TM) Client VM (build 1.5.0_10-b03, mixed mode, sharing)


C:\Documents and Settings\tchau>


Thanks for your reply. I have change the version.dat as suggested by Kirill.
It was a strict migration and I have done nothing else to modify the configuration especially the path settings, we still get the same error in our tomcat logs, the above is the environemnt settings from the command line and it shows it is currently running under java 1.5.

Where should I look at next?

BR,
Tom

0

Hello Tom,

   Please try the following:

   - If you start TeamCity with windows service, stop the service, backup and remove all logs from TeamCity/log directory, and try starting it using TeamCity/bin/runAll.bat start command
     Would it help? Please investigate logs for visible problems or send the archive of the logs directory to us.

   - Another thing to try is to setup TeamCity into a new directory and start it from new location. If this would work, that means some corruption during migration process.
   - Try changing data directory for TeamCity to check whether the problem is connected with configuration data. You can see TEAMCITY_DATA_PATH env. variable for that.

   Sorry for vague instructions, the problem is weird and we didn't face anything like this before.

   Regards,
   KIR

0

Hi,

1)

Thanks for your reply.I have attached log files here for investigation.

Logs.zip: are the log files generated after migration, I have place log files before migration under logs\archive. (The migration was done on 5th Dec 08)

Migration Logs.zip: are the logs file from migration, there are duplication because I was trying the correct format of the data path.

runAll logs.zip: are the files generated from runAll.bat under bin folder.

Please kindly be noticed that runAll.bat started up but I was not able to use the browser to connect to the server, I have tried to connect via port 80, 8000, 8080, 9000, 9090 etc.

I have placed the prompt from the CMD under runAll logs.zip, as CMD1.txt and CMD2.txt, where CMD1.txt is just how I start and stop the runAll.bat, and CMD2.txt is the display from the CMD while running runAll.bat.

2)

As this is a production server, I will try not to have multiple install on the server at the moment, but I will check with my team if I can do so.

3)

I dont see an environment variable as TEAMCITY_DATA_PATH ? Somthing similiar I can find is:

TEAMCITY_SERVER_OPTS

-agentlib:yjpagent

Thanks again for everything, your help is very much appreciated.

Best regards,
Thomas



Attachment(s):
runAll logs.zip
Migration Logs.zip
logs.zip
0

As this is a production server, I will try not to have multiple install on the server at the moment, but I will check with my team if I can do so.


My team agree I dont have a choice and I have tried another 2 new installation to verify the problem by now, as follow:

1.) First I tried to copy D:\TeamCity\.buildServer folder to D:\TeamCity2\.buildServer (this .buildServer folder was backuped before migration), then I start installing teamcity 4.0, and configure it to run as service on port 8000. This approach fails and I receive an error as HTTP Status 404 (please refer to the first post).

2.) And I then tried to install TeamCity 4.0 to D:\TeamCity3 without putting the D:\TeamCity\.buildServer folder in, and teamcity is able to start up with licence agreemetn page then to config server page, which looks fine to me.

So TeamCity is able to start properly without the repository, however, I wonder why I cannot load the repository in that was backed up before migration?

BR,
Thomas

0

1) All migration logs you've sent are from an unsuccessful runs. Either 'no driver' or 'can't connect' in all of them. Please send us proper logs. However I suspect that migration is irrelevant to the error. See item #3.

2) If in some strange case you're unsure which port is configured on server it's displayed on server startup, i.e
>INFO: Initializing Coyote HTTP/1.1 on http-8000
in your logs, also you can check TCP port's it's listiening to by various OS means.

3) Contents of your environment variable CLASSPATH looks suspicious to me - it will definetely clash with server startup shell script. Please try running script from environment with this variable unset.

0

Thanks guys. After all my team decided to rebuild the TeamCity server from scatch and point to MS SQL 2005 as a final solution to this problem.
It saves our effort here and reduce the downtime of the production environment.

Thanks again for everyone. Your help is very much appreciated.

Kind regards,
Thomas

0

Please sign in to leave a comment.