Agent has Docker Build Runner but my build configuration says there are no compatible agents

Hi Luis,

Please, make sure that the docker is actually installed on the agent, it's running and an agent user can execute docker commands. To check this, try to log in on the agent machine and execute simple `docker info` command under the same user as the TeamCity agent.

> The agent says it has the Docker Build Runner.

This just means that the agent has the required agent-side plugin part to run Docker-related steps.

I have the same problem as Luis. I have docker installed and when I run a test build step which calls docker version, it correctly logs the version to the build log but it still says "Unmet requirements: docker.server.version exists".

If I look under the agent parameters/configuration parameters, it shows docker.version with the correct value but not docker.server.version. How are these parameters created? Can I just add it manually somewhere?

Hi Luke! 

Could you tell me more about your setup? Could it be that you hit the same problem as described here https://youtrack.jetbrains.com/issue/TW-52266? 

Hi Nadia, The post is about sudo on Linux and we are running on Windows Server. We are running 2019.1 (build 65998) and I installed Docker Desktop for Windows manually but have not done anything else to set anything up. The Docker plugin is bundled and enabled but I'm not sure what the plugin does compared to Docker Desktop.

Thanks

By default on Windows Server, only Administrator users are able to execute docker commands. Could you please try to run `docker info` under the same user as your TeamCity agent runs and share the result? Seems, that you just need to allow other users on Windows to use docker.

Hi, we've run into the same problem. We have 5 build agents and two of these no longer have a docker.server.version in their configuration parameters, only a docker.version. It seems to me that the docker build runner specifically needs the docker.server.version parameter?

Hi Sander and Dushant. Could you please execute `docker info` command on the machine under the same user as a TeamCity agent and attach here the output? 

Hi Nadia,

PFB info.

I had  install TeamCity through the .rar file in my GCP VM and access that VM through my local machine and also I have  install teamcity on my local machine but it doesn't give me any agent error or above error that i have mention.

I think i don't have any enough permission to run docker command in virtual machine, but i can execute using SUDO.

Dushant, now TeamCity doesn't support running Docker commands using `sudo` but we have the related feature request in progress. Please, watch\vote for it. It's going to be fixed and released in the next bugfix release.

Hi Nadia,

This is the docker info of the build agent that does work:

-----------------

Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 27
Server Version: 18.09.2
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 9754871865f7fe2f4e74d43e2fc7ccd237edcbce
runc version: 09c8266bf2fcf9519a651b04ae54c967b9ab86ec
init version: fec3683
Security Options:
seccomp
Profile: default
Kernel Version: 4.9.125-linuxkit
Operating System: Docker for Windows
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 1.934GiB
Name: linuxkit-00155d38d607
ID: SDEF:VNMZ:I7IK:SLOK:5XNU:XEUD:TTMI:LXVE:EVMJ:DOAS:5JTF:MVFN
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
File Descriptors: 23
Goroutines: 50
System Time: 2019-07-18T06:41:15.9416188Z
EventsListeners: 2
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine

-----------------

Running docker info on an agent that doesn't work is tricky, I updated the docker engine and now it is borked. Rebooting the machine delivers some information:

Docker Engine - Community
Version: 18.09.3
API Version: 1.39
Go version: go1.10.8
Git commit: 774a1f4

Just for kicks I uninstalled the docker engine on one of the machines that was borked and reinstalled it. And now it works again. De docker version and build shown in the GUI are the same as on the machine that doesn't like to play nice.I can't run docker --version on that machine because the docker host isn't connected so that sucks. I do have the complete diagnostics of that machine however, if that helps.

Anyway, it seems that reinstalling the docker desktop version does solve this problem...

Hi Sander,

beyond what Nadia already mentioned, this is a limitation on docker itself and how it installs on linux, by default under admin rights. There are ways to work around the issue which are explained directly on docker's website: https://docs.docker.com/install/linux/linux-postinstall/

Please check those, as it's not unlikely that your misbehaving installations might be facing that issue.

Hi

I have the same issue after updated to 2019.1 (build 65998).

An agent with docker version 18.09.2 still works.

Another Centos 6 agent with older docker version 1.7.1 didn't work any more.

I suppose new teamcity version try to set variables from docker info? No server info for older docker version.

[agentaccount@xxxx ~]$ docker -v
Docker version 1.7.1, build 786b29d

[agentaccount@xxxx ~]$ docker info
Containers: 17
Images: 317
Storage Driver: devicemapper
Pool Name: docker-253:0-2883961-pool
Pool Blocksize: 65.54 kB
Backing Filesystem: extfs
Data file: /dev/loop0
Metadata file: /dev/loop1
Data Space Used: 18.81 GB
Data Space Total: 107.4 GB
Data Space Available: 45.19 GB
Metadata Space Used: 23.58 MB
Metadata Space Total: 2.147 GB
Metadata Space Available: 2.124 GB
Udev Sync Supported: true
Deferred Removal Enabled: false
Data loop file: /var/lib/docker/devicemapper/devicemapper/data
Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
Library Version: 1.02.117-RHEL6 (2016-12-13)
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 2.6.32-696.10.3.el6.x86_64
Operating System: <unknown>
CPUs: 2
Total Memory: 3.854 GiB
Name: xxxxx
ID: 7XCW:3EVE:3JHA:2H7E:2JNG:OSGC:ACBG:4OFF:UHVK:WQ6D:UGUS:ZHAB

Hi Denis,

Well, the installation is on Windows, so this probably isn't the exact issue as on Linux. But Docker for Windows is a bit...eh, special. But since we only have 5 build agents and primarily develop C# / .Net Framwork stuff I am hesitant to create a separate Linux build agent for Docker builds...

Hello,

   If build agent had been started before the docker was initialized, build agent won't report docker availability, because it checks for docker presence during the startup. Could it be the case?

  Build agent reports diagnostics about the process of docker detection to buildAgent/logs/teamcity-agent.log file, could also be helpful (grep for DOCKER line).

  Hope this helps,

I was able to reproduce the problem and created the issue in our bug tracker. Please watch\vote for it to get further updates. 

Buy the way could you please describe why do you use such an outdated docker? Is docker update isn't an option for you? 

It is just an old Centos 6 agent.

Update docker on the old box seems not very straight forward.

A Centos 7 agent works fine, but will need some time to replace all old agents.

I posted this on https://teamcity-support.jetbrains.com/hc/en-us/community/posts/115000628470-Docker-integration-plugin as well, but I see a number of posters here having the same issue.  I don't know about this issue on Linux, but here's my experience and solution on Windows.

TeamCity Server, TeamCity Build Agent and Docker Engine all run as services.  I think the problem occurs when the TeamCity Server and Build Agent services are up and running before the Docker service.  So maybe TeamCity is querying for the Docker information before Docker has finished starting up.

I had the problem just now.  I restarted the TeamCity services (Docker was now already running and settled) and the problem went away.

I tested by shutting down Docker Engine, then restarting the TeamCity services. Sure enough, I get that the docker.server.version requirement hasn't been met. Leaving TeamCity services running and restarting Docker Engine didn't help. I had to wait for Docker Engine to be running and then start TeamCity Server and Build Agent afterward. This has worked every time so far.

This agrees with some of the suggestions that the issue could be that Docker wasn't running and so many responses saying it was.  It's just that it wasn't fully operational at the time TeamCity queried the information.

I'm going to try the "Automatic (Delayed Start)" option for the TeamCity services in Windows Services to see if this makes life easier.

I've hit this too and yep, it's the delayed start issue for me (permissions are correct, docker is up to date).

Our setup uses systemd. I fixed it by adjusting the docker.service file to use the recommended startup flags (which for some reason our install wasn't - probably related to upstream churn), namely, "docker -H fd://" with "Type: notify". Then I adjusted the teamcity.service file to have "After: docker.socket" in it. Rebooting the machine yielded a surprising result - the docker properties still were not there, BUT, after a minute or two, they do reliably appear. 

I noticed that with this new setup running "docker info" on first boot is incredibly slow. My guess is that TC Agent runs it in a background thread, and this command then blocks waiting for dockerd to start up. So it does get there in the end, it is just a bit misleading because it looks like it failed at first.

With these config changes I resolved the problem. Hope that helps!

If I'm not mistaken, the problem should be less critical after the fix of https://youtrack.jetbrains.com/issue/TW-60588.

I.e. TeamCity agent should re-check for the docker presence periodically since TC 2019.2.1.