升级至Cisco NXOS 9.3(9)版本后Ansible SSH连接故障排查求助

我最近碰到一个棘手的问题，想请大家帮忙分析下：

环境信息 #

• Ansible服务器：Ansible Core 2.11.7，基于Python 3.6.8
• 目标设备：Cisco NXOS N93180LC交换机
• 原连接方式：通过paramiko进行SSH用户名/密码认证，此前一直正常

问题现象 #

之前交换机运行9.3(8)版本时，Ansible连接完全正常，但升级到9.3(9)版本后就彻底连不上了：

• 把交换机降级回9.3(8)，Ansible的SSH连接立刻恢复正常
• 用相同的SSH用户名/密码，通过Putty能正常连接这个9.3(9)的交换机；用Ansible连接其他非9.3(9)版本的NXOS设备也没问题
• 两种测试场景都失败：
  • 运行仅包含获取交换机主机名任务的Playbook（这个Playbook在其他设备上能正常执行）
  • 直接在Ansible服务器执行ssh user@ip-of-switch命令，命令会一直卡在加载状态，无法完成连接

已尝试的排查/解决措施 #

我已经试过以下几种方法，但都没解决问题：

• 关闭主机密钥检查：在ansible.cfg中设置ansible_host_key_checking = False
• 手动添加交换机SSH指纹：通过交换机的show ssh key命令获取指纹，手动添加到Ansible服务器的known_hosts文件中
• 切换Ansible的SSH连接后端：从paramiko改为libssh

错误日志 #

执行Playbook时的错误输出如下：

TASK [ask Hostname NXOS] ******************************************************************************************************************
task path: /home/ansible/var/test/Angel/nxos/playbook_hostcheck.yaml:24
The full traceback is:
Traceback (most recent call last):
  File "/home/ansible/.ansible/tmp/ansible-local-62419fhcrsg9/ansible-tmp-1653908906.3061478-6248-45658655329117/AnsiballZ_nxos_command.py", line 100, in <module>
    _ansiballz_main()
  File "/home/ansible/.ansible/tmp/ansible-local-62419fhcrsg9/ansible-tmp-1653908906.3061478-6248-45658655329117/AnsiballZ_nxos_command.py", line 92, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/home/ansible/.ansible/tmp/ansible-local-62419fhcrsg9/ansible-tmp-1653908906.3061478-6248-45658655329117/AnsiballZ_nxos_command.py", line 41, in invoke_module
    run_name='__main__', alter_sys=True)
  File "/usr/lib64/python3.6/runpy.py", line 205, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib64/python3.6/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_nxos_command_payload_xau7a6f5/ansible_nxos_command_payload.zip/ansible_collections/cisco/nxos/plugins/modules/nxos_command.py", line 237, in <module>
  File "/tmp/ansible_nxos_command_payload_xau7a6f5/ansible_nxos_command_payload.zip/ansible_collections/cisco/nxos/plugins/modules/nxos_command.py", line 206, in main
  File "/tmp/ansible_nxos_command_payload_xau7a6f5/ansible_nxos_command_payload.zip/ansible_collections/cisco/nxos/plugins/module_utils/network/nxos/nxos.py", line 1370, in run_commands
  File "/tmp/ansible_nxos_command_payload_xau7a6f5/ansible_nxos_command_payload.zip/ansible_collections/cisco/nxos/plugins/module_utils/network/nxos/nxos.py", line 129, in get_connection
  File "/tmp/ansible_nxos_command_payload_xau7a6f5/ansible_nxos_command_payload.zip/ansible/module_utils/connection.py", line 200, in __rpc__
ansible.module_utils.connection.ConnectionError: [Errno 110] Connection timed out
fatal: [17.237.73.11]: FAILED! => {
    "changed": false,
    "rc": 1
}
MSG: MODULE FAILURE
See stdout/stderr for the exact error
MODULE_STDERR:
Traceback (most recent call last):
  File "/home/ansible/.ansible/tmp/ansible-local-62419fhcrsg9/ansible-tmp-1653908906.3061478-6248-45658655329117/AnsiballZ_nxos_command.py", line 100, in <module>
    _ansiballz_main()
  File "/home/ansible/.ansible/tmp/ansible-local-62419fhcrsg9/ansible-tmp-1653908906.3061478-6248-45658655329117/AnsiballZ_nxos_command.py", line 92, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/home/ansible/.ansible/tmp/ansible-local-62419fhcrsg9/ansible-tmp-1653908906.3061478-6248-45658655329117/AnsiballZ_nxos_command.py", line 41, in invoke_module
    run_name='__main__', alter_sys=True)
  File "/usr/lib64/python3.6/runpy.py", line 205, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib64/python3.6/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_nxos_command_payload_xau7a6f5/ansible_nxos_command_payload.zip/ansible_collections/cisco/nxos/plugins/modules/nxos_command.py", line 237, in <module>
  File "/tmp/ansible_nxos_command_payload_xau7a6f5/ansible_nxos_command_payload.zip/ansible_collections/cisco/nxos/plugins/modules/nxos_command.py", line 206, in main
  File "/tmp/ansible_nxos_command_payload_xau7a6f5/ansible_nxos_command_payload.zip/ansible_collections/cisco/nxos/plugins/module_utils/network/nxos/nxos.py", line 1370, in run_commands
  File "/tmp/ansible_nxos_command_payload_xau7a6f5/ansible_nxos_command_payload.zip/ansible_collections/cisco/nxos/plugins/module_utils/network/nxos/nxos.py", line 129, in get_connection
  File "/tmp/ansible_nxos_command_payload_xau7a6f5/ansible_nxos_command_payload.zip/ansible/module_utils/connection.py", line 200, in __rpc__
ansible.module_utils.connection.ConnectionError: [Errno 110] Connection timed out

疑问与求助 #

我怀疑这是NXOS 9.3(9)版本和Ansible Core 2.11.7之间存在未被发现的兼容性Bug，但实在找不到其他排查方向了，恳请各位大佬提供技术帮助！

内容的提问来源于stack exchange，提问作者riteangle