TeamCity server crashing frequently
Hi,
Over the past few days I have started experiencing multiple server crashes per day. I have turned on all the debug logging in teamcity-server-log4j.xml, but that has not produced any helpful messages/stacktraces in the log files when the server crashes. I also tried increasing the heap size from 512m to 768m. The Java process just dies, and users see a 500 error message when trying to access TeamCity. I am running TC 4.5.4 (9071) on Mac OS X Server 10.5.8. Is there anything I can try to debug/fix this problem? Any clues would be appreciated.
Thanks!
--Steve
Please sign in to leave a comment.
There should be crash report file in the TeamCity process working directory. Something like: hs_err_pid2128.log
Please attach this file.
Hi Pavel,
Thanks for the reply. However, I don't see any log files with that sort of name anywhere... I did do some more digging in the other log files and am seeing this message in catalina.out that seems to correspond with our recent server crashes:
Invalid memory access of location 00000004 eip=013a9c15
I am going to run some memory tests on our server to see if that might be causing the crashes.
Thanks,
--Steve
You can also try to upgrade Java used by TeamCity.
We're running on JDK6 now, but the server process is still crashing. However, I no longer see the Invalid memory access messages in catalina.out when the server dies. Do you have any idea why I'm not able to find the crash report logs?
Hi!
You may update your server Tomcat and the JRE version, to find an adequate and supported combination.
Regards,
Olivier.
What version of JDK 6 do you use? Please try to install the latest one. It is hard to tell what is the reason of the crash without a crash log. As I told it usually is created in the Java process working directory.
Is it possible that crash occurs when someone opens web diff? If so try to disable colorer (it is a native library which can cause such crashes). See section "Turning Off Code Highlighting in the Web Diff": http://www.jetbrains.net/confluence/display/TCD4/Known+Issues#KnownIssues-TurningOffCodeHighlightingintheWebDiff
In TeamCity 5.0 we replaced colorer with pure Java library to avoid such crashes.