TeamCity Upgrade to 2020.2.3 fails to start on PostgreSQL

Completed

Youtrack link

We were running 2020.1.5 - we did an upgrade on the server to 2020.2.3. After the server started up this was the error in the logs:

```
[2021-03-18 08:42:51,789] ERROR - jetbrains.buildServer.STARTUP - Unexpected exception: SQL error when doing 'Querying for list of values'
ERROR: pgaudit stack is not empty while performing SQL query: SQL SELECT: select table_name from information_schema.tables where table_schema = current_schema() and table_type like '%TABLE%': org.postgresql.util.PSQLException: ERROR: pgaudit stack is not empty
[2021-03-18 08:42:51,789] INFO - jetbrains.buildServer.STARTUP - Current stage: TeamCity server startup error
```
The only different thing that we did from all the other upgrades we've done over the years is when we were on 2020.1.5 we migrated from MSSQL to PostgreSQL. The server has been running fine with that migration so far. This is the first upgrade we've done running on Postgresql.

We tried 2020.2.1, 2020.2.2 and both resulted in the same error on startup. We reinstalled 2020.1.5 and are running again.

I'm not sure even where to start looking at this. Thoughts? 

Here is a full snipped of the server startup log file:

```
[2021-03-18 08:42:39,517] INFO - jetbrains.buildServer.STARTUP - ===========================================================
[2021-03-18 08:42:39,517] INFO - jetbrains.buildServer.STARTUP - Starting TeamCity server
[2021-03-18 08:42:39,645] INFO - jetbrains.buildServer.STARTUP - TeamCity version: 2020.2.2 (build 85899), data format version 958
[2021-03-18 08:42:39,650] INFO - jetbrains.buildServer.STARTUP - OS: Windows Server 2019, version 10.0, amd64, Current user: DAL-BUILD01$, Time zone: CDT (UTC-05:00)
[2021-03-18 08:42:39,650] INFO - jetbrains.buildServer.STARTUP - Java: 11.0.9, OpenJDK 64-Bit Server VM (11.0.9+11-LTS, mixed mode, sharing), OpenJDK Runtime Environment (11.0.9+11-LTS), Amazon.com Inc.; JVM parameters: --add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED -XX:+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED -Djava.util.logging.config.file=c:\TeamCity\bin\..\conf\logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Xrs -Xmx1024m -Dteamcity.configuration.path=../conf/teamcity-startup.properties -Dlog4j.configuration=file:../conf/teamcity-server-log4j.xml -Dteamcity_logs=c:\TeamCity\bin\..\logs -Dignore.endorsed.dirs= -Dcatalina.base=c:\TeamCity\bin\.. -Dcatalina.home=c:\TeamCity\bin\.. -Djava.io.tmpdir=c:\TeamCity\bin\..\temp
[2021-03-18 08:42:39,651] INFO - jetbrains.buildServer.STARTUP - Garbage collection: G1 Young Generation, G1 Old Generation
[2021-03-18 08:42:39,652] INFO - jetbrains.buildServer.STARTUP - Web server version: Apache Tomcat/8.5.61
[2021-03-18 08:42:39,653] INFO - jetbrains.buildServer.STARTUP - Server IP address: 10.95.164.130
[2021-03-18 08:42:39,655] INFO - jetbrains.buildServer.STARTUP - Web server ports: 80 (protocol: HTTP/1.1, scheme: http), 8105 (shutdown port)
[2021-03-18 08:42:39,655] INFO - jetbrains.buildServer.STARTUP - Context path: "/", servlet real path: "C:\TeamCity\webapps\ROOT"
[2021-03-18 08:42:39,662] INFO - jetbrains.buildServer.STARTUP - TeamCity distribution type: exe
[2021-03-18 08:42:39,662] INFO - jetbrains.buildServer.STARTUP - TeamCity server node id: MAIN_SERVER
[2021-03-18 08:42:39,676] INFO - rocesses.ProcessTreeTerminator - Using jetbrains.buildServer.processes.ProcessTreeTerminatorWindows
[2021-03-18 08:42:39,682] INFO - .ProcessTreeTerminatorImplBase - Will use command 'C:\TeamCity\temp\TeamCity11572180758317119509ps\JetBrains.TeamCity.ps.x64.exe ps'.
[2021-03-18 08:42:39,835] INFO - jetbrains.buildServer.STARTUP - TeamCity server process ID (1160) stored in: C:\TeamCity\logs\teamcity.pid
[2021-03-18 08:42:39,835] INFO - jetbrains.buildServer.STARTUP - Version file lock was successfully obtained: C:\TeamCity\webapps\ROOT\BUILD_85899
[2021-03-18 08:42:39,851] INFO - jetbrains.buildServer.STARTUP - Current stage: Looking for the TeamCity Data Directory
[2021-03-18 08:42:39,851] INFO - jetbrains.buildServer.STARTUP - The TeamCity Data Directory path is determined using the startup configuration from 'C:\TeamCity\conf\teamcity-startup.properties' file
[2021-03-18 08:42:39,851] INFO - jetbrains.buildServer.STARTUP - The TeamCity Data Directory "C:\ProgramData\JetBrains\TeamCity" exists
[2021-03-18 08:42:39,851] INFO - jetbrains.buildServer.STARTUP - The main configuration file "C:\ProgramData\JetBrains\TeamCity\config\main-config.xml" exists
[2021-03-18 08:42:39,851] INFO - jetbrains.buildServer.STARTUP - The database properties file "C:\ProgramData\JetBrains\TeamCity\config\database.properties" exists
[2021-03-18 08:42:39,851] INFO - jetbrains.buildServer.STARTUP - The internal database data file "C:\ProgramData\JetBrains\TeamCity\system\buildserver.data" doesn't exist
[2021-03-18 08:42:39,866] INFO - jetbrains.buildServer.STARTUP - Data Directory version: 929
[2021-03-18 08:42:39,866] INFO - jetbrains.buildServer.STARTUP - Current stage: Looking for the database configuration
[2021-03-18 08:42:39,866] INFO - jetbrains.buildServer.STARTUP - Database properties file "C:\ProgramData\JetBrains\TeamCity\config\database.properties" exists
[2021-03-18 08:42:39,866] INFO - jetbrains.buildServer.STARTUP - Internal HSQL database file (C:\ProgramData\JetBrains\TeamCity\system\buildserver.data) doesn't exist
[2021-03-18 08:42:39,866] INFO - jetbrains.buildServer.STARTUP - Database connection URL: jdbc:postgresql://714a7093-29ba-4aed-9332-66c5ca3962f3.blijti4d0v0nkr55oei0.private.databases.appdomain.cloud:30013/ibmclouddb
[2021-03-18 08:42:39,866] INFO - jetbrains.buildServer.STARTUP - Using database connection URL from the database properties file. The URL is: jdbc:postgresql://714a7093-29ba-4aed-9332-66c5ca3962f3.blijti4d0v0nkr55oei0.private.databases.appdomain.cloud:30013/ibmclouddb
[2021-03-18 08:42:39,866] INFO - jetbrains.buildServer.STARTUP - Using external (POSTGRESQL) database
[2021-03-18 08:42:39,866] INFO - jetbrains.buildServer.STARTUP - Current stage: Connecting to the database
[2021-03-18 08:42:39,866] INFO - jetbrains.buildServer.STARTUP - JDBC loader: loading JDBC drivers (for the first time)
[2021-03-18 08:42:39,962] INFO - jetbrains.buildServer.STARTUP - Effective database connection properties:
[2021-03-18 08:42:39,962] INFO - jetbrains.buildServer.STARTUP - ApplicationName=TeamCity [defaults]
[2021-03-18 08:42:39,962] INFO - jetbrains.buildServer.STARTUP - autosave=always [defaults]
[2021-03-18 08:42:39,962] INFO - jetbrains.buildServer.STARTUP - cleanupSavepoints=true [defaults]
[2021-03-18 08:42:51,286] INFO - jetbrains.buildServer.STARTUP - Database character set: UTF8
[2021-03-18 08:42:51,286] INFO - jetbrains.buildServer.STARTUP - Database collation: C.UTF-8
[2021-03-18 08:42:51,414] INFO - jetbrains.buildServer.STARTUP - JDBC driver version: 42.2 (PostgreSQL JDBC Driver)
[2021-03-18 08:42:51,414] INFO - jetbrains.buildServer.STARTUP - Database system version: 12.5 (PostgreSQL)
[2021-03-18 08:42:51,414] INFO - jetbrains.buildServer.STARTUP - Connected to the database successfully
[2021-03-18 08:42:51,414] INFO - jetbrains.buildServer.STARTUP - Current stage: Checking the database
[2021-03-18 08:42:51,773] INFO - jetbrains.buildServer.STARTUP - Using schema: public
[2021-03-18 08:42:51,789] ERROR - jetbrains.buildServer.STARTUP - Unexpected exception: SQL error when doing 'Querying for list of values'
ERROR: pgaudit stack is not empty while performing SQL query: SQL SELECT: select table_name from information_schema.tables where table_schema = current_schema() and table_type like '%TABLE%': org.postgresql.util.PSQLException: ERROR: pgaudit stack is not empty
[2021-03-18 08:42:51,789] INFO - jetbrains.buildServer.STARTUP - Current stage: TeamCity server startup error
[2021-03-18 08:42:51,789] INFO - jetbrains.buildServer.STARTUP - Administrator can login from web UI using super user authentication token
[2021-03-18 08:42:51,789] INFO - jetbrains.buildServer.SERVER - Super user authentication token: 1893952278469228357 (use empty username with the token as the password to access the server)
[2021-03-18 08:46:07,253] INFO - jetbrains.buildServer.STARTUP - Server shutdown event received
[2021-03-18 08:46:07,253] INFO - jetbrains.buildServer.STARTUP - Current stage: TeamCity server is shutting down
[2021-03-18 08:46:07,253] INFO - jetbrains.buildServer.STARTUP - Stopping the main application
[2021-03-18 08:46:07,253] INFO - jetbrains.buildServer.STARTUP - The main servlet is not initialized, nothing to shut down.
[2021-03-18 08:46:07,258] INFO - jetbrains.buildServer.STARTUP - Version file lock was released: C:\TeamCity\webapps\ROOT\BUILD_85899
[2021-03-18 08:46:07,259] INFO - jetbrains.buildServer.STARTUP - Server shutdown complete. Uptime was 3m:27s
[2021-03-18 08:54:10,581] INFO - jetbrains.buildServer.STARTUP - Path to the startup configuration file: c:\TeamCity\bin\..\conf\teamcity-startup.properties
[2021-03-18 08:54:10,630] INFO - jetbrains.buildServer.SERVER - Using logs directory "C:\TeamCity\logs"
[2021-03-18 08:54:10,645] INFO - jetbrains.buildServer.STARTUP -
```

0
1 comment

As per the linked YouTrack issue, upgrading pgAudit fixed the issue.

0

Please sign in to leave a comment.