Agent fails build config "Process exited with code 127"
The build fails after entering the first and only step:
Step 1/1: Command Line
14:22:4 Running step within Docker container python:3
14:22:42 Starting: /bin/sh -c ". /opt/buildagent/temp/agentTmp/docker-wrapper-231835785673144342.sh && docker run --rm -w /opt/buildagent/work/3e7a09ce3025fbcf --label jetbrains.teamcity.buildId=524 -v "/opt/buildagent/lib:/opt/buildagent/lib:ro" -v "/opt/buildagent/tools:/opt/buildagent/tools:ro" -v "/opt/buildagent/plugins:/opt/buildagent/plugins:ro" -v "/opt/buildagent/work/3e7a09ce3025fbcf:/opt/buildagent/work/3e7a09ce3025fbcf" -v "/opt/buildagent/temp/agentTmp:/opt/buildagent/temp/agentTmp" -v "/opt/buildagent/temp/buildTmp:/opt/buildagent/temp/buildTmp" -v "/opt/buildagent/system:/opt/buildagent/system" --env-file /opt/buildagent/temp/agentTmp/docker-wrapper-1292889910090267051.envList --entrypoint /bin/sh "python:3" /opt/buildagent/temp/agentTmp/docker-shell-script-5726473492574467882.sh"
14:22:42 in directory: /opt/buildagent/work/3e7a09ce3025fbcf
14:22:42 /bin/sh: 0: Can't open /opt/buildagent/temp/agentTmp/docker-shell-script-5726473492574467882.sh
14:22:43 Process exited with code 127
14:22:43 Process exited with code 127 (Step: Command Line)
14:22:43 Step Command Line failed
The python:3 container, is mounting the agentTmp directory to the same place within itself (-v "/opt/buildagent/temp/agentTmp:/opt/buildagent/temp/agentTmp").
However, after inspecting the agent, we can see the the script is present, so it's not that the script is missing and all users have read permissions:
$ kubectl exec -it teamcity-agent-88466b89d-2phzf /bin/bash
root@teamcity-agent-88466b89d-2phzf:/# cd /opt/buildagent/temp/agentTmp/
root@teamcity-agent-88466b89d-2phzf:/opt/buildagent/temp/agentTmp# ls
build.finish.properties.gz build.start.properties.gz docker-shell-script-5726473492574467882.sh docker-wrapper-1292889910090267051.envList docker-wrapper-231835785673144342.sh
root@teamcity-agent-88466b89d-2phzf:/opt/buildagent/temp/agentTmp# ls -lah
total 20K
drwxr-xr-x 2 root root 217 Mar 12 14:22 .
drwxr-xr-x 7 root root 118 Mar 12 14:22 ..
-rw-r--r-- 1 root root 3.8K Mar 12 14:22 build.finish.properties.gz
-rw-r--r-- 1 root root 3.8K Mar 12 14:22 build.start.properties.gz
-rw-r--r-- 1 root root 110 Mar 12 14:22 docker-shell-script-5726473492574467882.sh
-rw-r--r-- 1 root root 249 Mar 12 14:22 docker-wrapper-1292889910090267051.envList
-rw-r--r-- 1 root root 788 Mar 12 14:22 docker-wrapper-231835785673144342.sh
root@teamcity-agent-88466b89d-2phzf:/opt/buildagent/temp/agentTmp#
For the sake of completeness, I also include our Kubernetes deployment config which runs the agent:
apiVersion: apps/v1
kind: Deployment
metadata:
name: teamcity-agent
spec:
replicas: 2
selector:
matchLabels:
name: teamcity-agent
strategy:
type: Recreate
template:
metadata:
labels:
name: teamcity-agent
spec:
containers:
- image: jetbrains/teamcity-agent:2019.1-linux
name: teamcity-agent
imagePullPolicy: "Always"
env:
- name: SERVER_URL
value: teamcity:8002
resources:
requests:
memory: 1G
limits:
memory: 4G
volumeMounts:
- name: dockersock
mountPath: "/var/run/docker.sock"
volumes:
- name: dockersock
hostPath:
path: "/var/run/docker.sock"
Historically, this has been working fine and this problem has only emerged recently. We're not aware of any change that could have caused this. Any light shed on this would be appreciated.
Please sign in to leave a comment.
Hi and sorry for the delay. Did you manage to find the cause for the problem? Is the agent being spawned via the kubernetes plugin as well or are you using something else for that?