TeamCity hangs every few days
Hi,
TeamCity version - TeamCity Professional 2024.12.2 (build 174504)
Linux - Ubuntu 24.04.2 LTS
Proxy - Nginx
Symptoms:
Every few days TeamCity server hangs (I can see 502 nginx error) and generating thread dump with header:
Server cannot shutdown properly: some non-daemon threads are still running
Upload id: 2025_02_13_p7dLbSXLbXsNP5Xv9UwVRH (file: 2025-02-13_07.11.10-shutdown.txt)
Any suggestions ?
Please sign in to leave a comment.
Hi,
For performance issues, such as slowness or delays, please collect a set of at least 10 thread dumps (spread across the issue timeframe) from the server or agent during the occurrence of the issue. You can refer to this guide for instructions. Ensure that you send us the dumps along with relevant data.
From the thread dump, we can observe the following details you mentioned:
Threads related to Tomcat (
http-nio-8111-exec-*
),Amazon EC2
, andJGit
are still active. These are likely waiting on network tasks, request processing, or cloud agent synchronization.Check the TeamCity Web UI for any running builds and manually stop them if necessary.
CPU Load Analysis
From the log file, I noted high CPU load due to the following details:
Memory Usage Analysis
While memory usage is not an immediate issue, prolonged garbage collection pressure could develop. The G1 Old Gen (45% usage) suggests that memory may not be efficiently reclaimed.
To optimize memory allocation, please follow the TeamCity Server Requirements and adjust the
-Xmx
memory value accordingly.Best Regards,
Tom
I forgot to tell that TeamCity server hangs after independently restarts.
,
Please collect a set of at least 10 thread dumps (spread across the issue timeframe) from the server or agent during the occurrence of the issue. You can refer to this guide (https://www.jetbrains.com/help/teamcity/reporting-issues.html#Server+Thread+Dump) for instructions.
Please also share the teamcity-server.log with us for further investigation.
Best Regards,
Tom