通过EC2跳板机从AWS Lambda访问SFTP的实现及报错排查求助

问题背景 #

构建AWS Lambda无服务器服务，用于每日与SFTP服务器进行文件上传下载操作。该SFTP服务器仅允许IP已加入白名单的AWS EC2实例访问。尝试使用Python的SSHTunnelForwarder和paramiko库实现SSH隧道转发+SFTP功能，但运行报错，需调试问题或提供其他实现方式（如Bash/JavaScript）。

现有Python代码 #

with SSHTunnelForwarder(
   (ssh_instance_ip, 22),
   ssh_username=ssh_user,
   ssh_private_key='<key_path>',
    remote_bind_address=('0.0.0.0', 22),
    local_bind_address=('127.0.0.1', 10022),
   ) as tunnel:

    tunnel.start()

    client = paramiko.SSHClient()
    client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    client.connect(password=sftp_pwd, username=sftp_user, hostname=sftp_host, port='<port_value>', allow_agent=True, disabled_algorithms=dict(pubkeys=["rsa-sha2-512", "rsa-sha2-256"]))

    tr = client.get_transport()
    tr.default_max_packet_size = 100000000
    tr.default_window_size = 100000000

    sftp = client.open_sftp()

    sftp.listdir('/')

    sftp.close

    client.close()

运行报错信息 #

...
DEBUG:paramiko.transport:Sending global request "keepalive@lag.net"
DEBUG:paramiko.transport:[chan 0] EOF sent (0)
DEBUG:paramiko.transport:EOF in transport thread
Traceback (most recent call last):
  File "/home/linuxbrew/.linuxbrew/Cellar/python@3.10/3.10.9/lib/python3.10/site-packages/paramiko/sftp_client.py", line 852, in _read_response
    t, data = self._read_packet()
  File "/home/linuxbrew/.linuxbrew/Cellar/python@3.10/3.10.9/lib/python3.10/site-packages/paramiko/sftp.py", line 201, in _read_packet
    x = self._read_all(4)
  File "/home/linuxbrew/.linuxbrew/Cellar/python@3.10/3.10.9/lib/python3.10/site-packages/paramiko/sftp.py", line 188, in _read_all
    raise EOFError()
EOFError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/work/scratch/tunnel_test/em_tunnel/app/tunnel_test.py", line 71, in <module>
    sftp.listdir('/')
  File "/home/linuxbrew/.linuxbrew/Cellar/python@3.10/3.10.9/lib/python3.10/site-packages/paramiko/sftp_client.py", line 218, in listdir
    return [f.filename for f in self.listdir_attr(path)]
  File "/home/linuxbrew/.linuxbrew/Cellar/python@3.10/3.10.9/lib/python3.10/site-packages/paramiko/sftp_client.py", line 239, in listdir_attr
    t, msg = self._request(CMD_OPENDIR, path)
  File "/home/linuxbrew/.linuxbrew/Cellar/python@3.10/3.10.9/lib/python3.10/site-packages/paramiko/sftp_client.py", line 822, in _request
    return self._read_response(num)
  File "/home/linuxbrew/.linuxbrew/Cellar/python@3.10/3.10.9/lib/python3.10/site-packages/paramiko/sftp_client.py", line 854, in _read_response
    raise SSHException("Server connection dropped: {}".format(e))
paramiko.ssh_exception.SSHException: Server connection dropped:

补充信息 #

• EC2实例通过user@host及RSA私钥访问
• SFTP服务器通过user@host及密码访问

问题调试与修正 #

现有代码存在几个关键错误，导致连接失败：

1. 隧道目标地址错误：remote_bind_address应设置为SFTP服务器的地址和端口，而非0.0.0.0:22，隧道的作用是将本地端口转发到EC2能访问的SFTP服务地址。
2. SFTP连接目标错误：通过隧道访问SFTP时，应连接本地绑定的地址127.0.0.1和端口10022，而非直接连接SFTP服务器地址。
3. 方法调用错误：sftp.close缺少括号，应改为sftp.close()。
4. Lambda环境私钥处理：Lambda中无法直接使用本地私钥路径，需将私钥内容存储在AWS Secrets Manager或环境变量中，读取字符串形式的私钥。

修正后的Python代码 #

import paramiko
from sshtunnel import SSHTunnelForwarder
import os
import io

# 从环境变量或Secrets Manager读取配置
ssh_instance_ip = os.environ['EC2_IP']
ssh_user = os.environ['EC2_USER']
ssh_private_key_content = os.environ['EC2_PRIVATE_KEY']  # 字符串形式的私钥
sftp_host = os.environ['SFTP_HOST']
sftp_port = int(os.environ['SFTP_PORT'])
sftp_user = os.environ['SFTP_USER']
sftp_pwd = os.environ['SFTP_PWD']

# 将私钥字符串转为paramiko的RSAKey对象
private_key = paramiko.RSAKey.from_private_key(io.StringIO(ssh_private_key_content))

with SSHTunnelForwarder(
    (ssh_instance_ip, 22),
    ssh_username=ssh_user,
    ssh_private_key=private_key,
    remote_bind_address=(sftp_host, sftp_port),  # 目标SFTP地址和端口
    local_bind_address=('127.0.0.1', 10022),
) as tunnel:
    # 隧道已自动启动，无需手动调用tunnel.start()

    client = paramiko.SSHClient()
    client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    # 连接本地隧道端口
    client.connect(
        hostname='127.0.0.1',
        port=tunnel.local_bind_port,
        username=sftp_user,
        password=sftp_pwd,
        allow_agent=False,  # Lambda环境禁用代理
        disabled_algorithms=dict(pubkeys=["rsa-sha2-512", "rsa-sha2-256"])
    )

    tr = client.get_transport()
    tr.default_max_packet_size = 100000000
    tr.default_window_size = 100000000

    sftp = client.open_sftp()
    print(sftp.listdir('/'))
    sftp.close()  # 修正括号
    client.close()

替代实现方式 #

Bash方式（Lambda中使用） #

需为Lambda创建包含ssh、sshpass、sftp的层，步骤如下：

1. 在Amazon Linux 2环境中编译打包所需工具。
2. 将工具压缩为zip包，上传为Lambda层。
3. Lambda代码中执行以下命令建立隧道并传输文件：

# 将EC2私钥写入临时文件
echo "$EC2_PRIVATE_KEY" > /tmp/ec2_key.pem
chmod 600 /tmp/ec2_key.pem

# 后台建立SSH隧道
ssh -i /tmp/ec2_key.pem -N -L 10022:$SFTP_HOST:$SFTP_PORT $EC2_USER@$EC2_IP &
TUNNEL_PID=$!

# 等待隧道建立
sleep 3

# 使用sftp传输文件，通过sshpass传入密码
sshpass -p "$SFTP_PWD" sftp -P 10022 $SFTP_USER@127.0.0.1 << EOF
ls /
get /remote/file.txt /tmp/local_file.txt
put /tmp/local_file.txt /remote/upload.txt
bye
EOF

# 关闭隧道
kill $TUNNEL_PID

JavaScript方式（使用ssh2库） #

使用ssh2和ssh2-sftp-client库实现隧道转发+SFTP：

const { Client: SSHClient } = require('ssh2');
const Client = require('ssh2-sftp-client');

// 配置信息
const config = {
  ec2: {
    host: process.env.EC2_IP,
    port: 22,
    username: process.env.EC2_USER,
    privateKey: process.env.EC2_PRIVATE_KEY
  },
  sftp: {
    host: '127.0.0.1',
    port: 10022,
    username: process.env.SFTP_USER,
    password: process.env.SFTP_PWD
  },
  remoteSftp: {
    host: process.env.SFTP_HOST,
    port: parseInt(process.env.SFTP_PORT)
  }
};

async function run() {
  const tunnel = new SSHClient();
  
  await new Promise((resolve, reject) => {
    tunnel.on('ready', resolve);
    tunnel.on('error', reject);
    // 建立到EC2的隧道，转发本地10022到SFTP服务器
    tunnel.connect({
      ...config.ec2,
      forwardOut: (_, __, dstHost, dstPort, callback) => {
        tunnel.outgoing(dstHost, dstPort, callback);
      }
    });
  });

  // 创建SFTP客户端，连接本地隧道端口
  const sftp = new Client();
  await sftp.connect(config.sftp);
  
  // 执行SFTP操作
  const files = await sftp.list('/');
  console.log(files);
  
  await sftp.end();
  tunnel.end();
}

run().catch(err => console.error(err));

内容的提问来源于stack exchange，提问作者felipe_franceschini