Windows Server 2019 agent: Unable to start ssh agent
I'm trying to upgrade our build agents from Windows Server 2016 to 2019 (preferrably with docker preinstalled). We've been deploying agents using an ARM-template in Azure for years with a custom install script. I simply changed from 2016 to 2019 in the template and deployed. No errors, and the new agent registeres successfully with Team City. However, when I try to run a build, I get errors from ssh-agent.
The first I got was: stderr: unable to start ssh-agent service, error :1058
I tried various things to fix this, among other things making sure that the ssh-agent service runs with a user account that also runs the agent service. Then I got this error instead:
Failed to start SSH Agent: java.lang.RuntimeException: Cannot parse ssh-agent output: ''
Using the option with Docker preinstalled gives you two different locations with OpenSSH in PATH (one in System and one in Program Files). The Docker option also has a service named OpenSSH Authentication Agent (with service name "ssh-agent"). This is the srevice I made run with a user account, which lead to the java.lang.RuntimeException.
Installing without the Docker option does not have this service and only one location in path (the System one).
In addition, we install Git for Windows which comes with it's own ssh-agent. This is the one I get from running "Get-Command ssh-agent.exe" on our old agents. Making sure this is the version used on the new agents, does not fix the issue. I still see error number 1.
We are using version 2018.2.4 of Team City.
Please sign in to leave a comment.
Hi Kristian,
are you launching the ssh-agent on your own, or are you trying to run it via the ssh agent build feature?
I am not aware of specific issues with windows 2019, but we do have one issue in our tracker with Windows 10 starting on a specific build, so it's not unlikely that they share the base and thus the problem: https://youtrack.jetbrains.net/issue/TW-55164
I've already tried the suggested solution in the issue you link to. Maybe I was a little vague when I wrote "Making sure this is the verson used on the new agents" in my post, but what I meant is that I removed all other locations of ssh from PATH, except the one from Git for Windows.
Hi Kristian,
thanks for reporting back. First of all, a short reminder that removing the content from the PATH variable needs to be done in a way that is picked up by the build agent. That is, on the user that runs its process and making sure that it's properly picked up upon start of the agent.
About the first error message you get, there seems to be a wide array of issues being caused since the ssh-agent was integrated into Windows, so I'm afraid there isn't much we can do here. I'd recommend trying to get in touch with the windows support in that regard, as it doesn't seem to be teamcity specific. If you can only replicate it in Teamcity on the other hand, please open a specific issue in our tracker for it.
If you already checked that, I'm afraid there isn't much we can do. If it worked in 2016 and it doesn't with the rest being the same in 2019, it seems like there might be some incompatibility. As described in the issue, TeamCity expects some output and it seems like it doesn't have any. Please report it on the tracker as well, as that's the way we will be able to help. The more information (Windows build number, architecture, etc), the better.
Yes, I've ensured that the PATH variable has been picked up. At least I think so. Changed it system wide and restarted the agent service.
But, I think you are right in that there is something weird in how our agents are configured. I guess all we can really do is keep trying. I'll try to remember to report back here when we solve or work around the problem :)
Hi Kristian,
if you can't find anything, I'd recommend opening an issue in our tracker for windows 2019 explicitly, as there might be something going on there, but I have to say that I haven't found any other reference to it.
This is indeed an issue with 2019. Using OpenSSH feature does not work which adds ssh-agent.exe to C:\windows\System32\OpenSSH which does not work with TeamCity. Using the one with Git still works however. It would be nice if there was an obvious answer to why teamcity cant leverage this.
For anyone who is encountering this issue on a relatively fresh install of server 2019, even with agents in dockers I was able to get this resolved. I installed openssh separately, in my case I used chocolately during my docker image build, but I'm sure other options work. I also then disabled the built in openssh that ships with windows server 2019 using commands found here https://docs.microsoft.com/en-us/windows-server/administration/openssh/openssh_install_firstuse
Hope that helps, since JetBrains themselves couldn't offer any!