"UnexpectedDBException" and "Server communication failure" after upgrade to 9.0
Hello,
Just upgraded teamcity version from 8.0.5 to 9.0 and sometimes builds fails with the following exception:
[17:04:10][Step 1/2] Unexpected exception occurred during message processing [17:04:10][Step 1/2] jetbrains.buildServer.serverSide.db.UnexpectedDBException: Unexpected exception SQLDataException/HsqlException: SQL error when doing: DML query SQL query: update stats set test_count=?, status_text=? where build_id=? SQL exception: data exception: string data, right truncation at org.hsqldb.jdbc.JDBCUtil.sqlException(JDBCUtil.java:322) at org.hsqldb.jdbc.JDBCUtil.sqlException(JDBCUtil.java:247) at org.hsqldb.jdbc.JDBCPreparedStatement.fetchResult(JDBCPreparedStatement.java:4655) at org.hsqldb.jdbc.JDBCPreparedStatement.executeUpdate(JDBCPreparedStatement.java:310) at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105) at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105) at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105) at jetbrains.buildServer.serverSide.db.DBFunctions$9.run(DBFunctions.java:921) at jetbrains.buildServer.serverSide.db.DBFunctions$9.run(DBFunctions.java:918) at jetbrains.buildServer.serverSide.db.DBInternalUtils.rawOperation(DBInternalUtils.java:72) at jetbrains.buildServer.serverSide.db.DBFunctions.executeDmlInternal(DBFunctions.java:917) at jetbrains.buildServer.serverSide.db.DBFunctions.executeDml(DBFunctions.java:875) at jetbrains.buildServer.serverSide.impl.StatusInfo$3.run(StatusInfo.java:2) at jetbrains.buildServer.serverSide.db.DBFacade$3.run(DBFacade.java:219) at jetbrains.buildServer.serverSide.db.DBFacade$3.run(DBFacade.java:217) at jetbrains.buildServer.serverSide.db.DBFacade.withDB(DBFacade.java:234) at jetbrains.buildServer.serverSide.db.DBFacade.withDB(DBFacade.java:217) at jetbrains.buildServer.serverSide.impl.StatusInfo.saveToDatabase(StatusInfo.java:34) at jetbrains.buildServer.serverSide.impl.RunningBuildImpl.setBuildStatusDescription(RunningBuildImpl.java:597) at jetbrains.buildServer.messages.serviceMessages.BuildStatusTranslator.translate(BuildStatusTranslator.java:13) at jetbrains.buildServer.messages.BuildServiceMessagesTranslator$MessagesProcessor.processServiceMessage(BuildServiceMessagesTranslator.java:12) at jetbrains.buildServer.messages.serviceMessages.ServiceMessagesProcessor$1.serviceMessage(ServiceMessagesProcessor.java:65) at jetbrains.buildServer.messages.serviceMessages.ServiceMessage.parse(ServiceMessage.java:137) at jetbrains.buildServer.messages.serviceMessages.ServiceMessagesProcessor.processTextMessage(ServiceMessagesProcessor.java:57) at jetbrains.buildServer.messages.BuildServiceMessagesTranslator.translateMessages(BuildServiceMessagesTranslator.java:5) at jetbrains.buildServer.messages.BuildMessagesProcessor.translateMessages(BuildMessagesProcessor.java:1) at jetbrains.buildServer.serverSide.impl.RunningBuildImpl.processMessage(RunningBuildImpl.java:486) at jetbrains.buildServer.serverSide.impl.RunningBuildImpl.addBuildMessages(RunningBuildImpl.java:7) at jetbrains.buildServer.serverSide.impl.RunningBuildImpl.access$100(RunningBuildImpl.java:229) at jetbrains.buildServer.serverSide.impl.RunningBuildImpl$2.doInTransactionWithoutResult(RunningBuildImpl.java:2) at org.springframework.transaction.support.TransactionCallbackWithoutResult.doInTransaction(TransactionCallbackWithoutResult.java:34) at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:133) at jetbrains.buildServer.serverSide.impl.RunningBuildImpl.addAgentBuildMessages(RunningBuildImpl.java:180) at jetbrains.buildServer.serverSide.impl.BuildAgentMessagesQueue.doProcessMessages(BuildAgentMessagesQueue.java:27) at jetbrains.buildServer.serverSide.impl.BuildAgentMessagesQueue.access$700(BuildAgentMessagesQueue.java:13) at jetbrains.buildServer.serverSide.impl.BuildAgentMessagesQueue$BuildMessagesQueue$1.run(BuildAgentMessagesQueue.java:1) at jetbrains.buildServer.util.NamedThreadFactory.executeWithNewThreadName(NamedThreadFactory.java:89) at jetbrains.buildServer.serverSide.impl.BuildAgentMessagesQueue$BuildMessagesQueue.processMessages(BuildAgentMessagesQueue.java:25) at jetbrains.buildServer.serverSide.impl.BuildAgentMessagesQueue$2.run(BuildAgentMessagesQueue.java:8) at java.lang.Thread.run(Thread.java:745) Caused by: java.sql.SQLDataException: data exception: string data, right truncation ... 40 more Caused by: org.hsqldb.HsqlException: data exception: string data, right truncation at org.hsqldb.error.Error.error(Error.java:136) at org.hsqldb.error.Error.error(Error.java:106) at org.hsqldb.types.CharacterType.castOrConvertToType(CharacterType.java:485) at org.hsqldb.types.CharacterType.convertToType(CharacterType.java:598) at org.hsqldb.StatementDML.getUpdatedData(StatementDML.java:661) at org.hsqldb.StatementDML.executeUpdateStatement(StatementDML.java:515) at org.hsqldb.StatementDML.getResult(StatementDML.java:197) at org.hsqldb.StatementDMQL.execute(StatementDMQL.java:195) at org.hsqldb.Session.executeCompiledStatement(Session.java:1378) at org.hsqldb.Session.execute(Session.java:991) at org.hsqldb.jdbc.JDBCPreparedStatement.fetchResult(JDBCPreparedStatement.java:4647) ... 37 more
Also, when build is running and I open build log, all messages are loaded correctly, but when new messages supposed to show up, I get the following message:
Server communication failure Server is unavailable since 12/24/2014, 10:02:39 AM Server stopped or communication with the server is not possible due to network failure.
After refreshing build log page, new messages shows up, but I get the same message, when newer messages wants to load.
Any ideas what could be wrong?
EDIT: Ok, those issues appears just sometimes, but not everytime. Strange..
Please sign in to leave a comment.
Hello,
The issue has been fixed in TeamCity 9.0.1, which is going to be released in the begining of the next week.
BTW it is not recommended to use internal HSQLDB database for production TeamCity instances. Please consider migration to an external database.
Migrated data from internal to MSSQL DB and updated to 9.0.1 version.
Seems like everything is ok now.