Podman启动的服务无法从外部网络访问问题排查求助

环境信息 #

• 机器A：Ubuntu 20.04.6 LTS，已安装Docker 23.0.1、Podman 3.4.2
• 通过Compose文件部署Prometheus实例，需求是让机器B能通过浏览器访问9090端口
• 服务已分配到外部创建的test网络：

Docker网络列表 #

docker network ls
NETWORK ID     NAME      DRIVER    SCOPE
1af699f798cd   bridge    bridge    local
47abbcc1fb96   host      host      local
bd34621bee08   none      null      local
1540ec6b02de   test      bridge    local

Podman网络列表 #

podman network ls
NETWORK ID    NAME        VERSION     PLUGINS
2f259bab93aa  podman      0.4.0       bridge,portmap,firewall,tuning
7867d86a50ef  test        0.4.0       bridge,portmap,firewall,tuning,dnsname

Docker启动的正常表现 #

使用Docker启动后，机器B可正常访问Prometheus，机器A上netstat显示端口监听正常：

sudo netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
...
tcp        0      0 0.0.0.0:9090            0.0.0.0:*               LISTEN      420189/docker-proxy
...
tcp6       0      0 :::9090                 :::*                    LISTEN      420196/docker-proxy
...

Podman启动的异常表现 #

使用Podman启动后，机器B无法访问Prometheus页面，但机器A上netstat显示9090端口仍处于监听状态：

sudo netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
...
tcp6       0      0 :::9090                 :::*                    LISTEN      444958/containers-r
...

通过ps命令确认相关进程：

sudo ps -ef | grep -E "444958|444523|443277"
abc       443277  419940  0 14:16 pts/0    00:00:00 /usr/bin/python3 /usr/local/bin/podman-compose up prometheus
abc       444523  443277  0 14:16 pts/0    00:00:00 podman start -a prometheus
abc       444524  443277  0 14:16 pts/0    00:00:00 sed -e s/^/?[1;34m[prometheus] |?[0m\ /;
abc       444676  444523  0 14:16 pts/0    00:00:00 /usr/bin/slirp4netns --disable-host-loopback --mtu=65520 --enable-sandbox --enable-seccomp -c -r 3 --netns-type=path /run/user/1001/netns/rootless-cni-ns tap0
abc       444958  444523  0 14:16 pts/0    00:00:00 containers-rootlessport
abc       444963  444958  0 14:16 pts/0    00:00:00 containers-rootlessport-child
abc       445067  420369  0 14:18 pts/1    00:00:00 grep --color=auto -E 444958|444523|443277

尝试sudo启动Podman的结果 #

根据Podman官方文档，无根模式启动的容器无法从外部访问，因此尝试用sudo启动，但问题依旧：

sudo netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
...
tcp        0      0 0.0.0.0:9090            0.0.0.0:*               LISTEN      454803/conmon
...

确认进程信息：

sudo ps -ef | grep 454803
root      454803       1  0 14:21 ?        00:00:00 /usr/libexec/podman/conmon --api-version 1 -c 2f60ecafd8cb76e41e5abb958e662f33c348608bedf9f5406687104e834f9922 -u 2f60ecafd8cb76e41e5abb958e662f33c348608bedf9f5406687104e834f9922 -r /usr/bin/crun -b /var/lib/containers/storage/overlay-containers/2f60ecafd8cb76e41e5abb958e662f33c348608bedf9f5406687104e834f9922/userdata -p /run/containers/storage/overlay-containers/2f60ecafd8cb76e41e5abb958e662f33c348608bedf9f5406687104e834f9922/userdata/pidfile -n prometheus --exit-dir /run/libpod/exits --full-attach -s -l journald --log-level warning --runtime-arg --log-format=json --runtime-arg --log --runtime-arg=/run/containers/storage/overlay-containers/2f60ecafd8cb76e41e5abb958e662f33c348608bedf9f5406687104e834f9922/userdata/oci-log --conmon-pidfile /run/containers/storage/overlay-containers/2f60ecafd8cb76e41e5abb958e662f33c348608bedf9f5406687104e834f9922/userdata/conmon.pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /var/lib/containers/storage --exit-command-arg --runroot --exit-command-arg /run/containers/storage --exit-command-arg --log-level --exit-command-arg warning --exit-command-arg --cgroup-manager --exit-command-arg systemd --exit-command-arg --tmpdir --exit-command-arg /run/libpod --exit-command-arg --runtime --exit-command-arg crun --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg --storage-opt --exit-command-arg overlay.mountopt=nodev,metacopy=on --exit-command-arg --events-backend --exit-command-arg journald --exit-command-arg container --exit-command-arg cleanup --exit-command-arg 2f60ecafd8cb76e41e5abb958e662f33c348608bedf9f5406687104e834f9922
nobody    454806  454803  0 14:21 ?        00:00:00 /bin/prometheus --config.file=/app.cfg/prometheus.yml --storage.tsdb.path=/app.cfg/prometheus_data --web.console.libraries=/usr/share/prometheus/console_libraries --web.console.templates=/usr/share/prometheus/consoles

补充测试 #

在机器A上执行curl localhost:9090，无论是Docker还是Podman启动的服务都能正常返回结果；由于Docker使用相同配置可正常被外部访问，排除防火墙问题。

问题 #

Podman启动的服务无法被外部机器访问的原因可能是什么？如需更多信息可提供。

内容的提问来源于stack exchange，提问作者Christian