docker.server.version doesnt exist for official agent image 2020.1

Answered

I just updated my TeamCity server to 2020.1, and thought i would also update the agents to 2020.1. However, build steps that run in a docker container are now saying there are no compatible agents with `Unmet requirements: docker.server.version exists`.

Going back to the 2019.2.4 version fixes the problem, and looking at the agent description in TC shows that indeed there is no value for `docker.server.version` in the 2020.1 version.

 

Did something change with the image?

7 comments
Comment actions Permalink

Hey - from TeamCity UI you can click Agents tab, then select your agent, then there's an Agent Logs tab. I'd say in teamcity-agent.log there'd be something logged.

0
Comment actions Permalink

This is what i found:

[2020-05-26 16:31:31,832]   INFO - rains.buildServer.AGENT.DOCKER - Docker client is available: 19.03.3
[2020-05-26 16:31:31,937]   INFO - rains.buildServer.AGENT.DOCKER - Docker server info is not available. Check whether docker daemon exists and it is up and running.
[2020-05-26 16:31:31,937]   INFO - rains.buildServer.AGENT.DOCKER - Docker-compose is not available. Check whether it has been installed and PATH environment variable contains path to it.

 

docker-support is included in the "plugin initialization completed" list and appears to have been successfully downloaded from the server.

0
Comment actions Permalink

Hi Francesco,

 

when an agent starts, teamcity checks connection to the docker service and pulls some information from it. The error you see means that either the service was not running yet, or that the user running the agent process does not have permissions to connect to it.

 

The most common causes for this are that the full server is restarting and the docker service starts after the build agent is already up, and that the initial setup by docker did not set the service in a way that the agent can communicate with it (typically set under admin/root accounts while teamcity's build agent does not have permissions for it)

0
Comment actions Permalink

That makes sense, but this works perfectly fine in the older version of teamcity, with identical setup. Also, it is running in mesos as a docker image, so docker is inherently already running.

Did something change with how teamcity connects to the local docker daemon?

0
Comment actions Permalink

Hi Francesco,

 

the only changes that have happened recently to the connection to docker services is that the default docker images do not run the process as root any longer, and as such, they don't have permission to connect to the docker daemon. This was already the case before for local docker installations and was a docker issue, not strictly related to teamcity (https://docs.docker.com/engine/install/linux-postinstall/), but if you are using the official images you might be newly impacted by it. More info here: https://www.jetbrains.com/help/teamcity/2020.1/upgrade-notes.html#UpgradeNotes-AgentDockerimagesrunundernon-rootuser

 

Can you check that?

0
Comment actions Permalink

I have the same problem after upgrading. Agent works in docker, image jetbrains/teamcity-agent. Before upgrading everything was fine.

One of the agent runs like this:

docker run -d \
--name peaceful_shannon \
--restart always \
--user $(id -u) \
--privileged -e DOCKER_IN_DOCKER=start \
-e SERVER_URL="https://some.site" \
-e AGENT_NAME="Home Server" \
-v /home/home/configs/teamcity-agent-conf:/data/teamcity_agent/conf \
-v /opt/buildagent/work:/opt/buildagent/work \
-v /opt/buildagent/temp:/opt/buildagent/temp \
-v /opt/buildagent/tools:/opt/buildagent/tools \
-v /opt/buildagent/plugins:/opt/buildagent/plugins \
-v /opt/buildagent/system:/opt/buildagent/system \
-v /var/run/docker.sock:/var/run/docker.sock \
jetbrains/teamcity-agent

Removing or adding "--privileged -e DOCKER_IN_DOCKER=start \" line does not affect.

Running `docker info` in agent container by user buildagent user (default user in jetbrains/teamcity-agent) causes an error:

ERROR: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.40/info: dial unix /var/run/docker.sock: connect: permission denied
errors pretty printing info

Running by root inside the container works fine.

Logs:[2020-06-26 05:47:50,689] INFO - rains.buildServer.AGENT.DOCKER - Docker client is available: 19.03.9,
[2020-06-26 05:47:50,775] INFO - rains.buildServer.AGENT.DOCKER - Docker server info is not available. Check whether docker daemon exists and it is up and running.,
[2020-06-26 05:47:50,775] INFO - rains.buildServer.AGENT.DOCKER - Docker-compose is not available. Check whether it has been installed and PATH environment variable contains path to it.

One more detail:

buildagent@e21ed8a30c71:/$ echo $DOCKER_IN_DOCKER
start
0
Comment actions Permalink

I found a resolution for my case. Docker GID (group ID) in the host system is 1001, meanwhile, docker GID in the container is 999.

Host system:

home@home:~$ ls -l /var/run/docker.sock
srw-rw---- 1 root docker 0 Jun 26 04:53 /var/run/docker.sock

Agent container:

buildagent@8e3cc53fdc6b:/$ ls -l /var/run/docker.sock 
srw-rw---- 1 root 1001 0 Jun 26 04:53 /var/run/docker.sock

I just add this line in docker run script:

  --group-add $(getent group docker | cut -d: -f3) \

which becomes

  --group-add 1001 \

After this, docker works for buildagent user inside docker.

1

Please sign in to leave a comment.