Python批量SSH操作Viptela设备执行命令报错求助

需求说明 #

• 批量SSH登录7台Viptela设备，首次登录自动确认主机密钥并存入known hosts
• 首次登录执行tools support touch_testbed触发设备重启
• 等待1分钟设备恢复后，再次登录执行5条命令（含切换到bash环境的操作）

手动操作流程 #

PS C:\Users\user1> ssh xyz@20.x.x.181
The authenticity of host '20.x.x.181 (20.x.x.181)' can't be established.
ECDSA key fingerprint is SHA256:R++eRV4YRmKsfMUr4BZ+Hx7gQmwW/dXeFsi4O6SybII.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '20.x.x.181' (ECDSA) to the list of known hosts.
viptela 20.9.999-sdwanaas-20.9-305

(xyz@20.x.x.181) Password:
Last login: Wed Aug 23 11:23:09 UTC 2023 from 128.x.x.180 on ssh
Welcome to Viptela CLI
User xyz last logged in 2023-08-23T04:06:20.207088+00:00, to ip-10-0-1-94, from 44.224.28.225 using netconf-ssh
xyz connected from 128.x.x.180 using ssh on vsmart-1
vsmart-1# tools support touch_testbed
File is touched (please login again)
Connection to 20.x.x.181 closed by remote host.
Connection to 20.x.x.181 closed.
PS C:\Users\user1>
PS C:\Users\user1> ssh xyz@20.x.x.181
viptela 20.9.999-sdwanaas-20.9-305

(xyz@20.x.x.181) Password:
Last login: Wed Aug 23 11:26:03 UTC 2023 from 128.x.x.180 on pts/0
Welcome to Viptela CLI
User xyz last logged in 2023-08-23T11:26:04.046308+00:00, to ip-10-0-1-96, from 128.x.x.180 using cli-ssh
xyz connected from 128.x.x.180 using ssh on vsmart-1
vsmart-1# tools internal ip_netns options "exec default bash"
vsmart-1:~# touch /etc/viptela/testbed
vsmart-1:~# touch /boot/testbed
vsmart-1:~# ls -l /etc/viptela/testbed;
-rw------- 1 root root 0 Aug 23 11:29 /etc/viptela/testbed
vsmart-1:~# ls -l /boot/testbed;
-rwx------ 1 root root 0 Aug 23 11:30 /boot/testbed
vsmart-1:~#

现有Paramiko脚本 #

import paramiko

hostname = []
list_of_ip = "20.x.x.181, 20.x.x.182, 20.x.x.183, 20.x.x.184, 20.x.x.185, 20.x.x.186, 20.x.x.187"
hostname=list_of_ip.split(",")
username = "xyz"
password = "abcdefghijk123456789"
cmd1="tools support touch_testbed"
cmd2='tools internal ip_netns options "exec default bash"'
cmd3="touch /etc/viptela/testbed"
cmd4="touch /boot/testbed"
cmd5="ls -l /etc/viptela/testbed;"
cmd6="ls -l /boot/testbed;"

#initialize the SSH client
client = paramiko.SSHClient()
#add to known hosts
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
#Login first time to execute the first command
for ip in hostname:
    print("ssh xyz@"+ip)
    try:
        client.connect(hostname=ip, username=username, password=password)
        #device will go for reboot after this first command
        stdin, stdout, stderr = client.exec_command(cmd1)
        #print the output of the command to check if it is successful
    except:
        print("[!] Cannot connect to the SSH Server")
        exit()
        
        
#Login second time to execute the remaining 5 commands
for ip in hostname:
    print("ssh xyz@"+ip)
    try:
        client.connect(hostname=ip, username=username, password=password)
        stdin, stdout, stderr = client.exec_command(cmd2)
        stdin, stdout, stderr = client.exec_command(cmd3)
        stdin, stdout, stderr = client.exec_command(cmd4)
        stdin, stdout, stderr = client.exec_command(cmd5)
        stdin, stdout, stderr = client.exec_command(cmd6)
        #print the output of the command to check if it is successful
    except:
        print("[!] Cannot connect to the SSH Server")
        exit()

报错信息 #

ssh xyz@20.x.x.181
ssh xyz@20.x.x.182
ssh xyz@20.x.x.183
ssh xyz@20.x.x.184
ssh xyz@20.x.x.185
ssh xyz@20.x.x.186
ssh xyz@20.x.x.187

Exception ignored in: <function BufferedFile.__del__ at 0x000001EB18C1ED30>
Traceback (most recent call last):
  File "C:\Users\user1\AppData\Local\Programs\Python\Python39\lib\site-packages\paramiko\file.py", line 67, in __del__
  File "C:\Users\user1\AppData\Local\Programs\Python\Python39\lib\site-packages\paramiko\channel.py", line 1390, in close
  File "C:\Users\user1\AppData\Local\Programs\Python\Python39\lib\site-packages\paramiko\channel.py", line 989, in shutdown_write
  File "C:\Users\user1\AppData\Local\Programs\Python\Python39\lib\site-packages\paramiko\channel.py", line 965, in shutdown
  File "C:\Users\user1\AppData\Local\Programs\Python\Python39\lib\site-packages\paramiko\transport.py", line 1920, in _send_user_message
AttributeError: 'NoneType' object has no attribute 'time'

问题分析与修复 #

核心问题 #

1. 复用SSHClient实例：单个SSHClient实例连续连接多台设备，未关闭旧连接，导致资源泄漏，触发BufferedFile销毁时的异常。
2. 未处理设备重启等待：执行touch_testbed后设备立即重启，脚本直接发起第二次连接，此时设备未恢复，连接大概率失败。
3. 交互式命令处理错误：tools internal ip_netns options "exec default bash"是切换到bash环境的交互式操作，exec_command每次会新建独立通道，后续命令不会在bash环境中执行，导致cmd3-cmd6失效。
4. 异常处理过于简陋：捕获所有异常直接退出，无法定位具体设备的问题。

修复后的脚本 #

import paramiko
import time
from paramiko.ssh_exception import SSHException, NoValidConnectionsError

# 设备列表（清理IP后的空格）
list_of_ip = "20.x.x.181,20.x.x.182,20.x.x.183,20.x.x.184,20.x.x.185,20.x.x.186,20.x.x.187"
hostnames = [ip.strip() for ip in list_of_ip.split(",")]
username = "xyz"
password = "abcdefghijk123456789"
cmd1 = "tools support touch_testbed"
# 交互式命令序列，包含切换bash和后续操作
post_reboot_cmds = [
    'tools internal ip_netns options "exec default bash"',
    "touch /etc/viptela/testbed",
    "touch /boot/testbed",
    "ls -l /etc/viptela/testbed",
    "ls -l /boot/testbed",
    "exit"  # 退出bash环境
]

def execute_initial_reboot(host):
    """首次登录执行重启命令"""
    client = paramiko.SSHClient()
    client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    try:
        print(f"[+] Connecting to {host} for initial reboot command")
        client.connect(host, username=username, password=password, timeout=10)
        stdin, stdout, stderr = client.exec_command(cmd1, timeout=5)
        # 读取命令输出
        output = stdout.read().decode().strip()
        error = stderr.read().decode().strip()
        if output:
            print(f"[+] {host} output: {output}")
        if error:
            print(f"[-] {host} error: {error}")
    except (SSHException, NoValidConnectionsError, TimeoutError) as e:
        print(f"[-] Failed to connect to {host}: {str(e)}")
    finally:
        # 确保关闭连接
        client.close()

def execute_post_reboot_commands(host):
    """重启后登录执行交互式命令"""
    client = paramiko.SSHClient()
    client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    try:
        print(f"\n[+] Connecting to {host} for post-reboot commands")
        # 重试连接，最多5次，每次间隔10秒
        for attempt in range(5):
            try:
                client.connect(host, username=username, password=password, timeout=10)
                break
            except (NoValidConnectionsError, TimeoutError):
                print(f"[-] {host} not ready, retrying {attempt+1}/5...")
                time.sleep(10)
        else:
            print(f"[-] {host} still unreachable after 5 attempts")
            return

        # 建立交互式shell会话
        shell = client.invoke_shell()
        time.sleep(2)  # 等待shell初始化

        # 发送命令序列
        for cmd in post_reboot_cmds:
            shell.send(cmd + "\n")
            time.sleep(1)  # 等待命令执行
            # 读取输出
            while shell.recv_ready():
                output = shell.recv(4096).decode().strip()
                print(f"[{host}] {output}")

    except SSHException as e:
        print(f"[-] SSH error on {host}: {str(e)}")
    finally:
        client.close()

# 第一步：批量执行重启命令
print("=== Executing initial reboot commands ===")
for host in hostnames:
    execute_initial_reboot(host)

# 等待设备重启，这里设置60秒
print("\n=== Waiting 60 seconds for devices to reboot ===")
time.sleep(60)

# 第二步：批量执行重启后的命令
print("\n=== Executing post-reboot commands ===")
for host in hostnames:
    execute_post_reboot_commands(host)

关键修复点 #

• 独立SSHClient实例：每个设备连接使用单独的SSHClient，用完立即关闭，避免资源泄漏。
• 连接重试机制：重启后重试连接设备，确保设备恢复后再执行命令。
• 交互式shell处理：使用invoke_shell建立会话，发送命令序列，确保切换bash后后续命令在正确环境执行。
• 完善的异常处理：捕获具体的SSH异常，输出详细错误信息，避免单个设备失败导致整个脚本退出。
• 输出读取：读取命令输出，方便验证操作结果。

内容的提问来源于stack exchange，提问作者Dipankar Nalui