Agent started but server shows message "1 cloud agents are starting" and agent goes to idle state
Hi, we (company) use TeamCity Enterprise 2024.07.3 (build 160765).
These are first/last few lines of docker agent startup/initialization/upgrade logs:
. . .
[2025-05-13 08:20:01,079] INFO - s.buildServer.agent.AgentMain2 - ===========================================================
[2025-05-13 08:20:01,097] INFO - s.buildServer.agent.AgentMain2 - TeamCity Build Agent 2024.07.3 (build 160765)
[2025-05-13 08:20:01,110] INFO - s.buildServer.agent.AgentMain2 - OS: Linux, version 5.15.0-200.131.27.el9uek.x86_64, amd64, Current user: buildagent, Time zone: BST (UTC+01:00)
[2025-05-13 08:20:01,112] INFO - s.buildServer.agent.AgentMain2 - Java: 1.8.0_412, OpenJDK 64-Bit Server VM (25.412-b08, mixed mode), OpenJDK Runtime Environment (1.8.0_412-b08), Amazon.com Inc.; JVM parameters: -ea -XX:+DisableAttachMechanism --add-opens=java.base/java.lang=ALL-UNNAMED -XX:+IgnoreUnrecognizedVMOptions -Xmx384m -Dteamcity_logs=../logs/ -Dlog4j2.configurationFile=file:../conf/teamcity-agent-log4j2.xml
[2025-05-13 08:20:01,113] INFO - s.buildServer.agent.AgentMain2 - Agent home is "/opt/buildagent"
[2025-05-13 08:20:01,113] INFO - s.buildServer.agent.AgentMain2 - Starting...
. . .
[2025-05-13 08:21:20,301] INFO - jetbrains.buildServer.AGENT - Downloading http://<host>/update/plugins/xray-teamcity-plugin-agent.zip ==> /opt/buildagent/update/plugins/xray-teamcity-plugin-agent.zip
[2025-05-13 08:21:20,852] INFO - jetbrains.buildServer.AGENT - The file /opt/buildagent/update/plugins/xray-teamcity-plugin-agent.zip has been downloaded successfully, file hash: SHA-1WzNxDsr9KByF/Q465HuDFeYoDKY=
[2025-05-13 08:21:20,852] INFO - jetbrains.buildServer.AGENT - Downloading http://<host>/update/buildAgent.zip ==> /opt/buildagent/update/buildAgent.zip
[2025-05-13 08:21:24,607] INFO - jetbrains.buildServer.AGENT - The file /opt/buildagent/update/buildAgent.zip has been downloaded successfully, file hash: SHA-1ACr2KjPdH6XmjlSACu2lAdnaitk=
[2025-05-13 08:21:25,021] INFO - rocesses.ProcessTreeTerminator - Using jetbrains.buildServer.processes.ProcessTreeTerminatorLinux
[2025-05-13 08:21:25,023] INFO - .ProcessTreeTerminatorImplBase - Collecting processes from PIDs current process
[2025-05-13 08:21:25,023] INFO - .ProcessTreeTerminatorImplBase - Will use command 'sh -c echo $$ && ps awwxo pid,ppid,command | tee'.
[2025-05-13 08:21:25,061] INFO - .ProcessTreeTerminatorImplBase - Will use command 'sh -c echo $$ && ps awwxo pid,ppid,command | tee'.
[2025-05-13 08:21:25,090] INFO - ses.ProcessTreeTerminatorLinux - Second thread id is 1048
[2025-05-13 08:21:25,090] INFO - ses.ProcessTreeTerminatorLinux - Thread is Process thread model: false
[2025-05-13 08:21:25,091] INFO - .ProcessTreeTerminatorImplBase - Current process PID 1048
[2025-05-13 08:21:25,092] INFO - .ProcessTreeTerminatorImplBase - No processes to kill
[2025-05-13 08:21:25,093] INFO - jetbrains.buildServer.AGENT - Exit for upgrade
[2025-05-13 08:21:25,094] INFO - ent.impl.upgrade.AgentExitCode - Agent exited. Upgrade process
I have 2 question regarding to this:
1. Even if we use same server/agent version the gui shows “Some plugins in cloud image are missing/outdated. This affects the time necessary to start the agent”. Why? Why the agent upgrades itself from the server even if we use same agent/server version?
2. When can we consider that the agent startup phase is finished and can pickup build from queue? According to docker agent log the last message from agent is “Agent exited. Upgrade process” and the agent remains running in docker container. Does it mean that the agent is up and can run build from queue? But why then the server still shows “1 cloud agent is starting” and the agent goes to idle state? It seems like the server does not receive notification from the agent, what ports should be opened on docker host where the agent is running and what on teamcity server?
Thanks & regards
Gabo.
Please sign in to leave a comment.
Hi,
Even if we use same server/agent version the gui shows “Some plugins in cloud image are missing/outdated. This affects the time necessary to start the agent”. Why? Why the agent upgrades itself from the server even if we use same agent/server version?
Even if the TeamCity core versions (i.e., buildAgent.jar and server build number) are the same, plugin versions may still differ between the server and the agent.
Why this happens:
- Cloud agents (e.g., Docker-based agents) are typically built once, with a static snapshot of the plugins at that time.
- When a cloud agent starts up, it connects to the TeamCity server.
- The server compares the plugin versions on the agent with its own.
- If any plugins are missing or outdated, the server instructs the agent to download the correct plugin versions via the /update/ endpoint, as seen in your logs:
- Downloading http://<host>/update/plugins/xray-teamcity-plugin-agent.zip
- Downloading http://<host>/update/buildAgent.zip
- This plugin synchronization causes the warning and can delay the agent's readiness.
When is an agent considered fully started and ready to pick up builds?
From the server's GUI, an agent is considered "registered and ready" when:
- The agent successfully connects to the server
- (If applicable) It is authorized.
- It enters the Idle state in the web UI (meaning it's ready to run builds).
You can confirm this process in the teamcity-server.log and teamcity-agent.log files by looking for entries like:
- Agent has connected to the server
- Agent registration complete
- Agent is authorized
- Agent is idle
Agent ↔ Server Communication Requirements
- The agent initiates the connection to the server.
- The default communication port is TCP 8111 (unless your TeamCity server is configured with a custom port).
- This port must be open from the Docker host (agent machine) to the TeamCity server.
- Ensure that any firewalls, network ACLs, or Docker network settings allow outbound traffic from the agent to the server on this port.
Best Regards,
Tom