gitlab ci runner with docker executor fails docker dind service's health check

I'm trying to run gitlab ci with a self-hosted runner with dokcer executor. I set docker dind as one of the services and it fails the health check with a message "nc: bad address 'docker'".

I set the runner as privileged mod, disabled tls, but still not working.

.gilab-ci.yml

stages:          # List of stages for jobs, and their order of execution
  - test

include:
  - local: '/chat-kit/chat-kit.gitlab-ci.yml'
    rules:
      - if: $CI_COMMIT_BRANCH == "main"
                    


test:
  stage: test
  script: echo "test"

chat-kit.gitlab-ci.yml

stages:
  - test

image: docker:28.0.4
services:
  - docker:28.0.4-dind

variables:
  DOCKER_TLS_CERTDIR: ""
  DOCKER_HOST: tcp://docker:2375

cache:
  paths:
    - .cache/pip

before_script:
  - wget .0.2/milvus-standalone-docker-compose.yml -O docker-compose.yml
  - docker compose up -d
  - cd chat-kit
  - python --version ; pip --version  # For debugging


pytest:
  stage: test
  script:
    - echo "test"

runner config

session_timeout = 1800

[[runners]]
  name = "dev-chat-ci-runner"
  url = "url"
  id = 17
  token = "token"
  token_obtained_at = 2025-04-01T04:51:02Z
  token_expires_at = 0001-01-01T00:00:00Z
  executor = "docker"
  [runners.cache]
    MaxUploadedArchiveSize = 0
    [runners.cache.s3]
    [runners.cache.gcs]
    [runners.cache.azure]
  [runners.docker]
    tls_verify = false
    image = "docker:28.0.4"
    privileged = true
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/certs/client", "/cache", "/var/run/docker.sock:/var/run/docker.sock"]
    shm_size = 0
    network_mtu = 0
    pull_policy = 'if-not-present'

This is the error log

parsing and downloads finished in 45 ms.
json schema validated in 101 ms
pytest starting docker:28.0.4 (test)
test   starting docker:28.0.4 (test)
test   copied to docker volumes in 1.26 s
pytest copied to docker volumes in 1.35 s
test   started service image: docker:28.0.4-dind with aliases: docker in 405 ms
pytest started service image: docker:28.0.4-dind with aliases: docker in 376 ms
test   > still running...
pytest > still running...
test   > still running...
pytest > still running...
test   > still running...
pytest > still running...
test   service image: docker:28.0.4-dind healthcheck failed
test     Command failed with exit code 143: docker run --rm --name=gcl-wait-for-it-579878-0-2375 --network 41d89441c11fab8f2f5278583228ff59023682bddab0edc889f1a55d5a06b90c docker.io/sumina46/wait-for-it docker:2375 -t 30
test     wait-for-it.sh: waiting 30 seconds for docker:2375
test     nc: bad address 'docker'
test     nc: bad address 'docker'
test     nc: bad address 'docker'
test     nc: bad address 'docker'
test     nc: bad address 'docker'
test     wait-for-it.sh: timeout occurred after waiting 30 seconds for docker:2375
test   also see (/home/atom/workspace/dev-chat/.gitlab-ci-local/services-output/test/docker:28.0.4-dind-0.log)
test   service image: docker:28.0.4-dind healthcheck failed
test     Command failed with exit code 143: docker run --rm --name=gcl-wait-for-it-579878-0-2376 --network 41d89441c11fab8f2f5278583228ff59023682bddab0edc889f1a55d5a06b90c docker.io/sumina46/wait-for-it docker:2376 -t 30
test     wait-for-it.sh: waiting 30 seconds for docker:2376
test     nc: bad address 'docker'
test     nc: bad address 'docker'
test     nc: bad address 'docker'
test     nc: bad address 'docker'
test     nc: bad address 'docker'
test     wait-for-it.sh: timeout occurred after waiting 30 seconds for docker:2376
test   also see (/home/atom/workspace/dev-chat/.gitlab-ci-local/services-output/test/docker:28.0.4-dind-0.log)
pytest service image: docker:28.0.4-dind healthcheck failed
pytest   Command failed with exit code 143: docker run --rm --name=gcl-wait-for-it-368789-0-2375 --network 2be139f73b236a037224f9eb6708ab460a858a74b732a879ea49abe12d274a27 docker.io/sumina46/wait-for-it docker:2375 -t 30
pytest   wait-for-it.sh: waiting 30 seconds for docker:2375
pytest   nc: bad address 'docker'
pytest   nc: bad address 'docker'
pytest   nc: bad address 'docker'
pytest   nc: bad address 'docker'
pytest   nc: bad address 'docker'
pytest   wait-for-it.sh: timeout occurred after waiting 30 seconds for docker:2375
pytest also see (/home/atom/workspace/dev-chat/.gitlab-ci-local/services-output/pytest/docker:28.0.4-dind-0.log)
pytest service image: docker:28.0.4-dind healthcheck failed
pytest   Command failed with exit code 143: docker run --rm --name=gcl-wait-for-it-368789-0-2376 --network 2be139f73b236a037224f9eb6708ab460a858a74b732a879ea49abe12d274a27 docker.io/sumina46/wait-for-it docker:2376 -t 30
pytest   wait-for-it.sh: waiting 30 seconds for docker:2376
pytest   nc: bad address 'docker'
pytest   nc: bad address 'docker'
pytest   nc: bad address 'docker'
pytest   nc: bad address 'docker'
pytest   nc: bad address 'docker'
pytest   wait-for-it.sh: timeout occurred after waiting 30 seconds for docker:2376
pytest also see (/home/atom/workspace/dev-chat/.gitlab-ci-local/services-output/pytest/docker:28.0.4-dind-0.log)
test   imported cache 'default' in 442 ms
test   $ wget .0.2/milvus-standalone-docker-compose.yml -O docker-compose.yml
test   > Connecting to github (20.200.245.247:443)
pytest imported cache 'default' in 815 ms
test   > Connecting to objects.githubusercontent (185.199.110.133:443)
pytest $ wget .0.2/milvus-standalone-docker-compose.yml -O docker-compose.yml
pytest > Connecting to github (20.200.245.247:443)
pytest > Connecting to objects.githubusercontent (185.199.110.133:443)
test   > saving to 'docker-compose.yml'
test   > docker-compose.yml   100% |********************************|  1303  0:00:00 ETA
test   > 'docker-compose.yml' saved
test   $ docker compose up -d
test   > time="2025-04-04T00:25:11Z" level=warning msg="/builds/support/dev-chat/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion"
test   > unable to get image 'minio/minio:RELEASE.2020-12-03T00-03-10Z': error during connect: Get "http://docker:2375/v1.48/images/minio/minio:RELEASE.2020-12-03T00-03-10Z/json": dial tcp: lookup docker on 127.0.0.11:53: server misbehaving
pytest > saving to 'docker-compose.yml'
pytest > docker-compose.yml   100% |********************************|  1303  0:00:00 ETA
pytest > 'docker-compose.yml' saved
pytest $ docker compose up -d
test   finished in 35 s  FAIL 1 
pytest > time="2025-04-04T00:25:11Z" level=warning msg="/builds/support/dev-chat/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion"
pytest > unable to get image 'minio/minio:RELEASE.2020-12-03T00-03-10Z': error during connect: Get "http://docker:2375/v1.48/images/minio/minio:RELEASE.2020-12-03T00-03-10Z/json": dial tcp: lookup docker on 127.0.0.11:53: server misbehaving
pytest finished in 35 s  FAIL 1 

 FAIL  pytest
  > 'docker-compose.yml' saved
  > time="2025-04-04T00:25:11Z" level=warning msg="/builds/support/dev-chat/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion"
  > unable to get image 'minio/minio:RELEASE.2020-12-03T00-03-10Z': error during connect: Get "http://docker:2375/v1.48/images/minio/minio:RELEASE.2020-12-03T00-03-10Z/json": dial tcp: lookup docker on 127.0.0.11:53: server misbehaving
 FAIL  test  
  > 'docker-compose.yml' saved
  > time="2025-04-04T00:25:11Z" level=warning msg="/builds/support/dev-chat/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion"
  > unable to get image 'minio/minio:RELEASE.2020-12-03T00-03-10Z': error during connect: Get "http://docker:2375/v1.48/images/minio/minio:RELEASE.2020-12-03T00-03-10Z/json": dial tcp: lookup docker on 127.0.0.11:53: server misbehaving
pipeline finished in 35 s

This is the log from docker dind services output

### stdout ###
nf_tables             385024 1441 nft_chain_nat,nft_limit,nft_compat
libcrc32c              12288  3 nf_nat,nf_conntrack,nf_tables
nfnetlink              20480  6 nf_conntrack_netlink,ip_set,nft_compat,nf_tables
ip_tables              32768  0 
x_tables               65536 14 xt_nat,xt_MASQUERADE,xt_set,ip6t_REJECT,xt_hl,ip6t_rt,ipt_REJECT,xt_LOG,xt_limit,xt_addrtype,xt_tcpudp,xt_conntrack,nft_compat,ip_tables
iptables v1.8.11 (nf_tables)
### stderr ###
cat: can't open '/proc/net/ip6_tables_names': No such file or directory
cat: can't open '/proc/net/arp_tables_names': No such file or directory
ip: can't find device 'nf_tables'
modprobe: can't change directory to '/lib/modules': No such file or directory
ip: can't find device 'ip_tables'
modprobe: can't change directory to '/lib/modules': No such file or directory
ip: can't find device 'ip6_tables'
modprobe: can't change directory to '/lib/modules': No such file or directory
mount: permission denied (are you root?)
Could not mount /sys/kernel/security.
AppArmor detection and --privileged mode might break.
mount: permission denied (are you root?)

The result of docker info on script.

pytest $ docker info
pytest > Client:
pytest >  Version:    28.0.4
pytest >  Context:    default
pytest >  Debug Mode: false
pytest >  Plugins:
pytest >   buildx: Docker Buildx (Docker Inc.)
pytest >     Version:  v0.22.0
pytest >     Path:     /usr/local/libexec/docker/cli-plugins/docker-buildx
pytest >   compose: Docker Compose (Docker Inc.)
pytest >     Version:  v2.34.0
pytest >     Path:     /usr/local/libexec/docker/cli-plugins/docker-compose
pytest > 
pytest > Server:
pytest > ERROR: error during connect: Get "http://docker:2375/v1.48/info": dial tcp: lookup docker on 127.0.0.11:53: server misbehaving
pytest > errors pretty printing info

科技改变生活-雨落星辰 - 所有的伟大,都源于一个勇敢的开始

gitlab ci runner with docker executor fails docker dind service's health check - Stack Overflow

与本文相关的文章

评论列表(0)