New instalation - build agent stuck in infinite upgrade loop
New clean instalation of TeamCity - TeamCity Professional 2017.2.3 (build 51047)
Agent is periodically disconnected, uppon inspecting the agent log I find and infinite loop of:
[2018-03-27 10:46:09,242] INFO - jetbrains.buildServer.AGENT - Exit for upgrade
(... upgrade)
[2018-03-27 10:46:49,716] WARN - gentUnpackingPluginJarSearcher - Failed to unpack plugin: "C:\TeamCity\buildAgent\plugins\idea.zip": jetbrains.buildServer.agent.plugins.files.PluginUnpackException: Failed to unpack entry from "C:\TeamCity\buildAgent\plugins\idea.zip" to "C:\TeamCity\buildAgent\tools\idea". Error unpacking "plugins/Kotlin/kotlinc/lib/kotlin-compiler.jar" to "C:\TeamCity\buildAgent\tools\idea\plugins\Kotlin\kotlinc\lib\kotlin-compiler.jar": java.util.zip.ZipException: invalid entry CRC (expected 0xd93c60f3 but got 0xe7c240e7)
[2018-03-27 10:47:00,142] WARN - ent.plugins.AgentPluginsLoader - Failed to unpack all plugins
[2018-03-27 10:47:00,142] WARN - ent.plugins.AgentPluginsLoader - ====> Agent will upgrade AGAIN
(... repeat)
This repeats many dozens of times - rendering the instalation completely unusable
Please sign in to leave a comment.
The corruption can be possibly caused by HW - I'm running memory diagnostics etc. to uncover this.
In any case - endless loop is not much helpful here. I'd expect few retries and then reporting problem to user in web GUI (e.g. "Agen failed - unable to upgrade - error XXX. Retry?")
Hi, and sorry for the delay.
The error message seems to imply that the file might have been corrupted during the installation. If other agents work alright, it's possible that this issue happened only on this agent, but it might as well be a problem with the server containing a corrupted file. Please trying replacing them. Also please consider that some third party tools like antivirus sometimes interrupt the unpack process and can lead to issues.
Leaving 1/2 as a reminder for myself the next time I upgrade my agent, and 1/2 for anyone else having the same issue. I have 8 agents on my server. 7 of them run as docker containers on the same physical box as the server. The 8th is a separate physical box. The docker agents always upgrade automatically after a server update - the standalone box NEVER does. It gets stuck in the "upgrade loop" similar to what is described above (my automatic agent upgrades actually have no errors or "Failed to..." warnings anywhere - but still insists on continually upgrading.
I'm sure it's a problem with the agent box/install, I just haven't quite figured out what yet (I'm learning toward file permissions). Here is what I do to fix the problem:
1) Stop the agent service
2) Copy the agent config to a safe place (<agent_home>/conf/buildAgent.properties)
3) (Re)move the agent home directory (for me is: `/opt/buildagent`)
4) Manually download the agent package (http://<teamcity_url>/update/buildAgent.zip) (can be downloaded without auth)
5) Unpack buildAgent.zip into clean home directory
6) Run install script with server url as first parameter (<agent_home>bin/install.sh <teamcity_server_url>)
7) Step 5 starts the agent - stop it again.
8) Copy agent config file from step to back into agent home.
8b) Ensure file permissions are correct for all of <agent_home> (may not apply to your install - it does for mine)
9) Start server - should come up without intervention from the server (no need to authorize agent)