We are using TeamCity since several years and we are running the latest release at all times (we typically upgrade within 5 days of any patch or new version release). This means that we currently run 9.1 (build 36973) which we upgraded about July 16th.
Already the day after the upgrade, and then every 3-5 days the TeamCity server stops. Or actually its server service stops. Without any valid reason, as far as I can tell. When I check in windows services the TeamCity Server service is clearly stopped. In the server.log file there is no explanation. And in Windows event viewer I get the following:
The description for Event ID 404 from source TeamCity (see below) cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.
If the event originated on another computer, the display information had to be saved with the event.
The following information was included with the event:
TeamCity JetBrains JetService v1.1.730.777
Service process exited without service stop request
This does not provide much lead on what is going on. Some installation details:
* OS is Win 2008 R2 SP1 64-bit
* We use SQL server as DB
* The windows service run with a domain user account (which also acts as local admin on the server)
* No configuration changes have been applied since quite many months
* some java config details:
Java version: 1.8.0_45
Java VM info: Java HotSpot(TM) Server VM (32 bit)
Java Home path: c:\TeamCity\jre
Server: Apache Tomcat/7.0.59
-Djava.util.logging.config.file=c:\TeamCity\bin\..\conf\logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Dteamcity.git.fetch.separate.process=false -Dteamcity.git.fetch.process.max.memory=1024M -Dteamcity.git.stream.file.threshold.mb=1024M -Xrs -Xmx1300m -XX:MaxPermSize=270m -Dteamcity.configuration.path=../conf/teamcity-startup.properties -Dlog4j.configuration=file:../conf/teamcity-server-log4j.xml -Dteamcity_logs=../logs/ -Djsse.enableSNIExtension=false -Djava.endorsed.dirs=c:\TeamCity\bin\..\endorsed -Dcatalina.base=c:\TeamCity\bin\.. -Dcatalina.home=c:\TeamCity\bin\.. -Djava.io.tmpdir=c:\TeamCity\bin\..\temp
* There is no build agent installed on server (however we have 12 agents in total, running in enterprise license)
* The server stop time is random and ometimes in morning, sometimes in afternoon and sometimes in evening (so far not during night time)
* We are running build tasks almost around the clock
* The solution is to start the service again and wait ten minutes for everything to initialize. However this requires that an admin user is monitoring and available.
I am not sure how to trace this issue further and what information that needs to be collected. If anyone have any ideas of what to test or suggestions on how to trouble shoot I would be quite happy. Currently the issue is mostly annoying since it is not very often, but if the error happens when no-one is monitoring it slows down our teams quite much.