PgBouncer无法连接Postgres Docker Compose服务的问题排查

问题背景 #

Docker Compose搭建的应用稳定运行数月后，突然出现PgBouncer无法连接Postgres的故障，报错：

pgbouncer_1 | 2023-11-02 22:54:28.949 UTC [1] LOG S-0x56100966f7a0: user/pw@(bad-af):0 closing because: server DNS lookup failed (age=0s)

已做操作：

• 多次卸载重装Docker、回滚镜像版本，无效
• 应用容器可ping通PgBouncer和Postgres；PgBouncer容器安装ping后也能连接同级容器
• 临时通过在pgbouncer服务的docker-compose配置中指定DNS resolver解决问题：

dns:
   - 8.8.8.8

环境信息：

• ankane/pgvector:v0.5.1
• bitnami/pgbouncer:1.21.0-debian-11-r4
• Docker version 24.0.7, build afdd53b
• Docker Compose version v2.23.0
• Debian 11（近期切换为systemd-resolved，已添加DNSStubListenerExtra=172.17.0.1配置）

Docker网络检查输出：

[
    {
        "Name": "backend-api_default",
        "Id": "3fdd76ff82ae2f3d98c184ed33ccc56244161125edc283fcc4f5d72ea33dbdf1",
        "Created": "2023-11-02T19:05:54.381213131-04:00",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "172.21.0.0/16",
                    "Gateway": "172.21.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "54f6f251fa0c9c71637aa8c2a4a559309b62ac45b3a93d6476f897442d8cef67": {
                "Name": "backend-api-postgres-exporter-1",
                "EndpointID": "6ba129ed346276cc0e7e354d30083945cd04eb6b5d3d233b0021662a46171bd9",
                "MacAddress": "02:42:ac:15:00:05",
                "IPv4Address": "172.21.0.5/16",
                "IPv6Address": ""
            },
            "64c8a17b4b554fb4fa8b0a5e625b621db3c56f350105b41fd23ec925be0e5fe2": {
                "Name": "backend-api-pgbouncer-1",
                "EndpointID": "685b17404212c480253fb5fa1750e2084c7d9cd2e708a54761c05b3b8c2b9eb4",
                "MacAddress": "02:42:ac:15:00:06",
                "IPv4Address": "172.21.0.6/16",
                "IPv6Address": ""
            },
            "8e2bee065f29459b16051eaa416396f1bb73558d9eb153c8635ba08c4de8ca5b": {
                "Name": "backend-api-api-1",
                "EndpointID": "e3672ca8154acc093ed91997067a16e7cfad8fd352b19fd455e0ead6ee5ae907",
                "MacAddress": "02:42:ac:15:00:07",
                "IPv4Address": "172.21.0.7/16",
                "IPv6Address": ""
            },
            "bdb093e7d4373e9abeaf9f35a0618dfc0abc828fa3a493367a163c5b3051f7bc": {
                "Name": "backend-api-grafana-1",
                "EndpointID": "fd4266e9dfab487ab04a19d33f6477e9051f5ec64a2974def1a24f58ccfbd7ad",
                "MacAddress": "02:42:ac:15:00:04",
                "IPv4Address": "172.21.0.4/16",
                "IPv6Address": ""
            },
            "e0f8d0ae29e8377cfc811c5bd0497f527a8511f14f6e03ef0e18788e9137e796": {
                "Name": "backend-api-prometheus-1",
                "EndpointID": "7785eb85093d7466d60927564c76b4f5f0eb5ccbc5627832980db76cab9ac9fe",
                "MacAddress": "02:42:ac:15:00:02",
                "IPv4Address": "172.21.0.2/16",
                "IPv6Address": ""
            },
            "f6743ebafd14641978be56c60a7f2b5f8bf849d282b221c52cbc4a8b09692cce": {
                "Name": "backend-api-postgres-1",
                "EndpointID": "30c10329252a76eb2965c4f2b1366b04d32644f4d752354607450f1946358832",
                "MacAddress": "02:42:ac:15:00:03",
                "IPv4Address": "172.21.0.3/16",
                "IPv6Address": ""
            }
        },
        "Options": {},
        "Labels": {
            "com.docker.compose.network": "default",
            "com.docker.compose.project": "backend-api",
            "com.docker.compose.version": "2.23.0"
        }
    }
]

原因分析 #

核心问题是Docker自定义bridge网络下的容器名DNS解析失效，结合切换到systemd-resolved的操作，大概率是以下情况：

1. systemd-resolved的DNSStubListenerExtra=172.17.0.1配置未正确生效，导致Docker容器默认使用的网关DNS无法解析容器名
2. Docker未正确配置为使用systemd-resolved提供的DNS服务，容器内部的/etc/resolv.conf指向的DNS服务器无法处理Docker网络的内部域名
3. PgBouncer的DNS解析逻辑依赖容器默认DNS，而默认DNS服务出现故障

排查步骤 #

1. 检查Docker全局DNS配置
   查看/etc/docker/daemon.json，确认是否配置了正确的DNS服务器：

   cat /etc/docker/daemon.json
   

   正常应该指向systemd-resolved的监听地址（比如172.17.0.1或127.0.0.53）

2. 验证systemd-resolved状态
   确认服务运行正常，且DNSStubListenerExtra配置已加载：

   systemctl status systemd-resolved
   resolvectl status
   

   检查输出中是否包含172.17.0.1的监听记录

3. 在PgBouncer容器内测试DNS解析
   进入容器，用nslookup或dig测试Postgres容器名的解析：

   docker exec -it backend-api-pgbouncer-1 bash
   nslookup backend-api-postgres-1
   dig backend-api-postgres-1
   

   正常应该返回Postgres的IP（172.21.0.3）

4. 检查容器的resolv.conf
   在PgBouncer容器内查看DNS配置：

   cat /etc/resolv.conf
   

   确认nameserver是Docker bridge网关地址（172.21.0.1），而不是其他无法解析内部域名的DNS

5. 主机端测试内部域名解析
   在Debian主机上测试是否能解析容器名：

   resolvectl query backend-api-postgres-1.backend-api_default
   

   正常应返回容器IP

修复方案 #

方案1：配置Docker全局DNS（推荐） #

修改/etc/docker/daemon.json，添加systemd-resolved的监听地址和备用DNS：

{
  "dns": ["172.17.0.1", "8.8.8.8"]
}

重启Docker服务：

systemctl restart docker

这样所有容器默认使用指定DNS，无需单独配置

方案2：修复systemd-resolved配置 #

确保/etc/systemd/resolved.conf或其.d目录下的配置文件包含：

[Resolve]
DNSStubListener=yes
DNSStubListenerExtra=172.17.0.1

重启systemd-resolved和Docker：

systemctl restart systemd-resolved
systemctl restart docker

让Docker容器使用systemd-resolved提供的网关DNS服务，正确解析内部容器名

方案3：固定PostgresIP并直接连接（应急） #

在docker-compose.yml中为Postgres服务配置固定IP：

services:
  postgres:
    image: ankane/pgvector:v0.5.1
    networks:
      default:
        ipv4_address: 172.21.0.3

然后修改PgBouncer的连接配置，直接使用IP而非容器名：

# PgBouncer配置文件中
host=172.21.0.3

此方法绕过DNS解析，但需注意容器重启后IP不会变化，适合临时应急

方案4：检查Docker网络的DNS设置 #

查看Docker网络的DNS配置，确认是否存在异常：

docker inspect backend-api_default | grep -i dns

如果网络未配置DNS，可尝试重新创建网络：

docker-compose down
docker network rm backend-api_default
docker-compose up -d

内容的提问来源于stack exchange，提问作者pdoherty926