Azure中GitLab Runner配置docker-autoscaler作业执行失败

已在Azure虚拟机规模集中配置GitLab Runner的docker-autoscaler执行器及fleeting插件，完成以下操作：

• 基于Ubuntu 22.04 LTS创建托管Azure VM镜像
• 安装Docker并将azureuser加入docker组
• 配置Docker随systemctl开机启动

目前Runner可触发规模集新增实例，但作业无法启动，报错：

ERROR: Job failed (system failure): Cannot connect to the Docker daemon at http://internal.tunnel.invalid. Is the docker daemon running? (docker.go:951:0s)

使用的版本信息 #

• Ubuntu 22.04 LTS
• Docker version 26.1.3, build b72abbb
• GitLab Runner Version: 17.0.0
• Git revision: 44feccdf
• Git branch: 17-0-stable
• GO version: go1.21.9
• Built: 2024-05-16T13:46:14+0000
• OS/Arch: linux/amd64

Runner的config.toml配置 #

concurrent = 1
check_interval = 0
connection_max_age = "15m0s"
shutdown_timeout = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "My Runner"
  url = "https://gitlab.com"
  id = 123123123
  token = "glrt-Some-Token"
  token_obtained_at = 2024-03-28T14:02:04Z
  token_expires_at = 0001-01-01T00:00:00Z
  executor = "docker-autoscaler"

  [runners.cache]
    MaxUploadedArchiveSize = 0

  [runners.docker]
    tls_verify = false
    image = "docker:stable"
    privileged = false
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/cache", "/certs/client"]
    shm_size = 0
    network_mtu = 0
    services_limit = -1

  # Autoscaler config
  [runners.autoscaler]
    plugin = "azure" # for >= 16.11, ensure you run `gitlab-runner fleeting install` to automatically install the plugin

    # for versions < 17.0, manually install the plugin and use:
    # plugin = "fleeting-plugin-azure"

    capacity_per_instance = 1
    max_use_count = 1
    max_instances = 10

    [runners.autoscaler.plugin_config] # plugin specific configuration (see plugin documentation)
      name = "gitlab-runner-scale-set"
      subscription_id = "SUBSCRIPTION_ID"
      resource_group_name = "gitlab-runner"

    [runners.autoscaler.connector_config]
      username = "azureuser"
      password = "SOME-PASSWORD"
      use_static_credentials = true
      timeout = "10m"
      use_external_addr = true

    [[runners.autoscaler.policy]]
      idle_count = 1
      idle_time = "10m0s"

遗漏的配置及修复步骤 #

1. 配置Docker守护进程监听TCP端口
   默认Docker仅监听Unix套接字，需修改配置让其同时监听TCP端口：

   • 创建或编辑/etc/docker/daemon.json，添加内容：

     {
       "hosts": ["unix:///var/run/docker.sock", "tcp://0.0.0.0:2375"]
     }
     

     若需TLS加密，改为"tcp://0.0.0.0:2376"并补充证书配置。
   • 重启Docker服务：sudo systemctl restart docker

2. 在Runner配置中指定Docker主机地址
   在[runners.autoscaler.connector_config]段添加Docker主机映射配置，利用SSH隧道转发访问实例内的Docker：

   [runners.autoscaler.connector_config]
     # ... 原有配置
     docker_host = "tcp://localhost:2375"
   

3. 开放Azure VM实例的防火墙端口
   在对应网络安全组（NSG）中添加入站规则，允许Runner所在网络访问实例的2375端口（无TLS）或2376端口（TLS），建议限制源IP范围以提升安全性。

4. 验证实例内Docker状态
   在新增的VM实例中执行docker ps确认Docker运行正常；执行curl http://localhost:2375/version确认TCP端口可访问。

5. 测试SSH连接与Docker权限
   手动测试SSH连接并验证Docker权限：ssh azureuser@<VM实例IP> docker ps，确保azureuser无需额外密码即可执行Docker命令。

内容的提问来源于stack exchange，提问作者Joaquín L. Robles