Agent Plugins Reluctance to update
We have written our own agent plugin to link our TeamCity commits fixed version information into the correct field in AzureDevOps. The plugin works great, what isn't so good is how TeamCity handles plugins and versioning.
Firstly, the Agent Plugins don't support version tag in the XML, for some very weird reason, so it's really not possible to see from the TeamCity webUI what version of our plugin is loaded on the server, “ready” to send to Agents.
More importantly however, the system for plugin updates is very buggy. Our agents are running different versions of the plugin, when I upload a new version, it doesn't automatically get sent to the agents and builds are kicked off using old and arbitrary versions of the plugin. This is really bad. I see bugs in the tracker about this, however for us, it's never worked correctly, including in the claimed fixed versions (We are running 2024.03.3 (build 156364) and TW-87921 claims to be fixed, it's not.
What's odd, is the agent knows something has changes, as the agent report as being “outdated”, however the plugin updates are VERY lazy, bordering on not at all, and builds will be launched with out of date plugins.
Sort of related, there is no way to update a plugin. You delete what you have on the server, and upload your new version.
What we end up doing is manually remoting into all our agents and copying the files, as it's the only way that works for us. This is really poor.
Is there a way to force plugin updates? As even doing a full reboot of the Agent OS isn't enough to make it do a plugin check. This really shouldn't be a thing, as really, how long does it take to compare a manifest of SHA checksums….
Please sign in to leave a comment.
I'm currently working around this by including the version in my plugin version mytool123 mytool124 etc and then updating the scripts that use these tools, and when nothing is using the older tools, removing it.
It's not ideal, and clearly not how things should need to work.
Glad to know that you had found a solution about this. But I am unable to reproduce it with TeamCity 2024.03.3.
If there is a new plugin uploaded, the connected agent will update it automatically. For more detailed information, please refer to the video link. link(https://drive.google.com/file/d/1QmNPw23dsuaYZTP0cEFUf4PticnRNw-y/view)
Could you please share the teamcity-agent.log (https://www.jetbrains.com/help/teamcity/viewing-build-agent-logs.html#Log+Files) to us for further investigation?
Files can be uploaded via https://uploads.jetbrains.com/. Please let us know the exact id after the upload.
Best Regards,
Tom
Unfortunately, I am unable to share the logs. I have tried this again here today, and it failed immediately.
How I recreated:
Result: 2nd build will still show it's using Tool Version A.
Sorry to know that still doesn't work for you.
> Immediately Kick off a new build of the same plan
Have you tried waiting for a short while? It seems the new version still needs some time to sync. Since you can't share the logs, we can't get more information to debug the issue.
Can't really "wait a while", as that is not how CI build management works. I have to be able to trust the TeamCity is going to use the correct version of the tool, and currently it's possible for several builds to fail before a tool finally gets updated on the agent.