Database corruption/Upgrade procedure

Hello Tim,

Tim McNerney wrote:

I was excited to see the new EAP become available to do some more testing.

I installed 1270, started it up and had a look for a while. Didn't have time to do much before having to shut it down. The next time I started it up, it wouldn't go. I'm getting some stack traces in logs/catalina.out that lead me to believe that perhaps the database got munged. Some entries:

I guess that before starting up 1270 you shut down previous version without
waiting for the database to shut down cleanly. And after the next hard shutdown
things became worse.

To restore database data, try to do the following:

- shutdown server
- in $HOME/.BuildServer/system, remove buildserver.data file and create file
with the same name and zero size.
- start server

This will result in lengthy process of database recovery from backup. If
you're lucky, the data will be restored.

If this will fail, or database exceptions will reappear, you can recreate
database. For this, simply remove 'system' subdirectory under .BuildServer (but
before make sure that you don't need artefacts which are placed into this
directory). Then start buildserver again - empty database wll be created.

java.lang.IndexOutOfBoundsException
at java.nio.Buffer.checkBounds(Buffer.java:454)
at java.nio.ByteBuffer.get(ByteBuffer.java:648)
at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:233)
at org.hsqldb.persist.NIOScaledRAFile.read(NIOScaledRAFile.java:218)
at org.hsqldb.persist.DataFileCache.readObject(DataFileCache.java:590)
at org.hsqldb.persist.DataFileCache.get(DataFileCache.java:541)
at org.hsqldb.Table$RowStore.get(Table.java:3453)
at org.hsqldb.Table.setIndexRoots(Table.java:1340)
at org.hsqldb.Table.setIndexRoots(Table.java:1370)
at org.hsqldb.DatabaseCommandInterpreter.processSet(DatabaseCommandInterpret
er.java:2290)
at org.hsqldb.DatabaseCommandInterpreter.executePart(DatabaseCommandInterpre
ter.java:305)
at org.hsqldb.DatabaseCommandInterpreter.execute(DatabaseCommandInterpreter.
java:169)
at org.hsqldb.Session.sqlExecuteDirectNoPreChecks(Session.java:975)
at org.hsqldb.scriptio.ScriptReaderText.readDDL(ScriptReaderText.java:93)
at org.hsqldb.scriptio.ScriptReaderBase.readAll(ScriptReaderBase.java:88)
at org.hsqldb.persist.Log.processScript(Log.java:615)
at org.hsqldb.persist.Log.open(Log.java:202)
at org.hsqldb.persist.Logger.openLog(Logger.java:113)
at org.hsqldb.Database.reopen(Database.java:301)
at org.hsqldb.Database.open(Database.java:257)
at org.hsqldb.DatabaseManager.getDatabase(DatabaseManager.java:215)
at org.hsqldb.DatabaseManager.newSession(DatabaseManager.java:138)
at org.hsqldb.jdbc.jdbcConnection.<init>(jdbcConnection.java:2434)
at org.hsqldb.jdbcDriver.getConnection(jdbcDriver.java:183)
at org.hsqldb.jdbcDriver.connect(jdbcDriver.java:161)
at java.sql.DriverManager.getConnection(DriverManager.java:525)
at java.sql.DriverManager.getConnection(DriverManager.java:140)
at jetbrains.buildServer.serverSide.impl.ConnectionPool$ConnectionCreator.createInstance(ConnectionPool.java:15)

.
.
java.sql.SQLException: error in script file line: 42 S1000 General error java.lang.IndexOutOfBoundsException in statement

There's more, but I think that should give some idea.

Along these lines, I wanted to ask what the recommended procedure when upgrading. Specifically:

- Do the BuildAgents all need to be updated?

They should be upgraded automatically. If not, this is a bug.

- Are there any files in ~/.BuildServer that should be deleted.

No. Actually, we deliver a set of converters which should upgrade your
database data.

- What about the config files. I noticed that they've been broken down by project now. Are there edits that need to be made due to the upgrade.

Upgrade of configuration files is done automatically. No manual editing
should be done (but is should be safe to edit files and they should be re-read
by the server).


Hope, this helps,

KIR

--
Kirill Maximov
Software Developer
JetBrains, Inc.
http://www.jetbrains.com
"Develop with pleasure!"

That's great. I just wiped my system directory and started fresh. Nothing serious lost. As long as I have my configs maintained, that's what is important. At least during evaluation/EAP.

And yes, I did shut down using "kill -9" at some point, because the server was not shutting down gracefully. I don't remember the specific circumstance, but this is something that is going to happen and protecting against it will be necessary.

Looking forward to trying out the new version and getting you some feedback.

--Tim