自托管Docker-GitLab实例SSH连接失败排查求助

我有一个自托管的Docker-GitLab实例（当前版本v16.8），此前正常运行多年，停用一年后重新启用并完成更新。GitLab网页、CI任务、容器仓库等功能均正常，但无法通过SSH与服务器进行Git交互，例如执行git clone ssh://git@INTERNETIP:10022/repo.git时失败。

本地机器连接的扩展错误日志 #

OpenSSH_8.9p1 Ubuntu-3ubuntu0.6, OpenSSL 3.0.2 15 Mar 2022
debug1: Reading configuration data /home/user/.ssh/config
debug1: /home/christian/.ssh/config line 1: Applying options for myurl.tld
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/home/user/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/home/user/.ssh/known_hosts2'
debug2: resolving "myurl.tld" port 10022
debug3: resolve_host: lookup myurl.tld:10022
debug3: ssh_connect_direct: entering
debug1: Connecting to myurl.tld [externalip] port 10022.
debug3: set_sock_tos: set socket 3 IP_TOS 0x10
debug1: connect to address externalip port 10022: Connection refused
ssh: connect to host myurl.tld port 10022: Connection refused
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

环境配置 #

• GitLab容器内SSH端口为22，外部映射为10022，gitlab.rb相关配置：

  external_url 'https://myurl.tld'
  gitlab_rails['gitlab_shell_ssh_port'] = 10022
  gitlab_sshd['enable'] = false # 也尝试过设为true（GitLab自研SSH服务）
  #gitlab_sshd['listen_address'] = '[::]:22' # 也尝试过设为10022
  

• Docker容器启动命令（省略挂载卷）：

  docker run \
  --detach \
  --name gitlab \
  --restart always \
  -p 10022:22 -p 10080:80 -p 10443:443 -p 4567:4567 \
  gitlab/gitlab-ce:latest
  

• GitLab容器连接默认Docker bridge网络，可正常通过外部端口10443访问HTTPS网页。
• 其他Docker容器可通过Nginx反向代理对外提供服务，示例配置：

  # nginx.conf
  server {
         listen INTERNETIP:443 ssl http2;
         server_name myurl.tld;
  ...
         location ~ ^/.* {
                  proxy_pass http://0.0.0.0:10443;
                  proxy_set_header Host              $host;
                  proxy_set_header X-Real-IP         $remote_addr;
                  proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
                  proxy_set_header X-Forwarded-Proto $scheme;
          }
  ...
  

• GitLab实例中添加的SSH和GPG密钥均有效，本地.ssh/config已配置，SSH代理可识别证书。
• 外部防火墙已开放10022端口，服务器本地无iptables拦截规则，甚至添加了测试规则-A INPUT -p tcp -m tcp --dport 10022 -j ACCEPT；停用容器后在该端口启动Python-SimpleHTTPServer可正常访问。
• 曾在GitLab容器内搭建第二个SSHD实例，将外部10022端口映射到该实例，但连接无响应，启动日志显示主机密钥不匹配：

  Public key for /etc/ssh/ssh_host_rsa_key does not match private key
  debug1: private host key #0: ssh-rsa SHA256:xxx
  Public key for /etc/ssh/ssh_host_ecdsa_key does not match private key
  debug1: private host key #1: ecdsa-sha2-nistp256 SHA256:xxx
  Public key for /etc/ssh/ssh_host_ed25519_key does not match private key
  debug1: private host key #2: ssh-ed25519 SHA256:xxx
  

已排查情况 #

• 在服务器本地执行git clone ssh://git@127.0.0.1:10022/repo.git、ssh -vvT git@172.17.0.3 -p 22（容器内部IP）或ssh -vvvT git@127.0.0.1 -p 10022均可成功连接并执行Git/SSH命令；
• 但在服务器本地使用公网IP或域名执行相同命令时，会出现与本地机器连接时相同的错误。

补充：Docker自动生成的当前iptables规则 #

-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-N DOCKER
-N DOCKER-ISOLATION-STAGE-1
-N DOCKER-ISOLATION-STAGE-2
-N DOCKER-USER
-N f2b-ssh
-A INPUT -p tcp -m tcp --dport 22 -j f2b-ssh
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A DOCKER -d 172.17.0.3/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 4567 -j ACCEPT
-A DOCKER -d 172.17.0.3/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 443 -j ACCEPT
-A DOCKER -d 172.17.0.2/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 80 -j ACCEPT
-A DOCKER -d 172.17.0.3/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 80 -j ACCEPT
-A DOCKER -d 172.17.0.3/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 22 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN
-A f2b-ssh -j RETURN

请问如何进一步测试和调试，找出阻碍我像一年前那样连接Git服务器的原因？

内容的提问来源于stack exchange，提问作者naletups