NestJS中使用@aws-sdk/client-s3上传S3时遇RequestTimeout问题求助

我在NestJS中间件中使用AWS SDK v3（@aws-sdk/client-s3）上传附件到Amazon S3时，持续遇到超时错误：

RequestTimeout: Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed.

已尝试更换AWS账号、调整包版本、验证凭证等操作，问题仍未解决。中间件代码如下：

import { BadRequestException } from '@nestjs/common';
import { Request, Response, NextFunction } from 'express';
import * as multer from 'multer';
import { S3Client, PutObjectCommand } from '@aws-sdk/client-s3';

const s3Client = new S3Client({
  region: process.env.AWS_REGION,
  credentials: {
    accessKeyId: process.env.AWS_ACCESS_KEY,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
  },
});

interface S3UploadMiddlewareOptions {
  fields: { name: string; maxCount?: number }[];
  fileFilter?: (
    req: Request,
    file: Express.Multer.File,
    cb: multer.FileFilterCallback,
  ) => void;
}

/**
 * A factory that returns an Express middleware for handling file uploads
 * with Multer (.fields) and then uploading each file to S3.
 *
 * @param fields  An array of { name, maxCount } for Multer .fields()
 * @returns       An Express middleware function
 */
export function createS3UploadMiddleware(options: S3UploadMiddlewareOptions) {
  console.log('inside createS3UploadMiddleware');
  console.log('options.fields', options.fields);
  console.log('options.fileFilter', options.fileFilter);

  const upload = multer({
    storage: multer.memoryStorage(),
    fileFilter: options.fileFilter ?? ((req, file, cb) => cb(null, true)),
  });

  return async (req: Request, res: Response, next: NextFunction) => {
    console.log('middleware called');
    upload.fields(options.fields)(req, res, async (err: any) => {
      console.log('upload.fields called');
      if (err) {
        console.log('error', err);
        return next(err);
      }

      console.log('req.files', req.files);
      if (!req.files || Object.keys(req.files).length === 0) {
        return next();
      }

      const filesMap = req.files as Record<string, Express.Multer.File[]>;
      const allFiles: Express.Multer.File[] = Object.values(filesMap).flat();

      const folderName = process.env.AWS_FOLDER_NAME || 'default-folder';

      try {
        console.log('uploading files to s3');
        const uploadPromises = allFiles.map(async (file) => {
          const key = `${folderName}/${Date.now()}-${file.originalname}`;
          await s3Client.send(
            new PutObjectCommand({
              Bucket: process.env.AWS_BUCKET_NAME,
              Key: key,
              Body: file.buffer,
              ContentType: file.mimetype,
            }),
          );

          const fileUrl = `https://${process.env.AWS_BUCKET_NAME}.s3.${process.env.AWS_REGION}.amazonaws.com/${key}`;
          return {
            fieldname: file.fieldname,
            originalname: file.originalname,
            mimetype: file.mimetype,
            size: file.size,
            location: fileUrl,
          };
        });

        const uploadedFiles = await Promise.all(uploadPromises);
        console.log('uploadedFiles', uploadedFiles);

        req.body.uploadedFiles = uploadedFiles;
        next();
      } catch (uploadErr) {
        console.log('upload error', uploadErr);
        return next(uploadErr);
      }
    });
  };
}

已尝试的排查步骤：

• 验证.env文件中的AWS凭证，更换过AWS账号
• 反复核对区域和存储桶名称（例如俄亥俄州的us-east-2）
• 测试不同大小的文件
• 尝试不同版本的@aws-sdk/client-s3包

以下是额外的排查与解决建议：

1. 显式配置S3客户端超时参数 #

AWS SDK v3默认超时可能较短，尤其是网络环境不佳时，可手动延长超时时间：

const s3Client = new S3Client({
  region: process.env.AWS_REGION,
  credentials: {
    accessKeyId: process.env.AWS_ACCESS_KEY,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
  },
  // 全局配置超时
  requestTimeout: 30000, // 请求超时30秒
  connectionTimeout: 10000, // 连接超时10秒
});

// 或单独为PutObjectCommand设置超时
await s3Client.send(
  new PutObjectCommand({/* 配置项 */}),
  { timeout: 30000 }
);

2. 排查网络与代理环境 #

若服务运行在有代理的环境（如内网、Docker容器），需为S3客户端配置代理：

import { NodeHttpHandler } from "@aws-sdk/node-http-handler";
import { HttpsProxyAgent } from "https-proxy-agent";

const s3Client = new S3Client({
  region: process.env.AWS_REGION,
  credentials: {/* 凭证 */},
  requestHandler: new NodeHttpHandler({
    httpAgent: new HttpsProxyAgent(process.env.HTTP_PROXY),
  }),
});

同时用curl测试服务到S3端点的连通性：

curl -v https://<你的存储桶名>.s3.us-east-2.amazonaws.com

3. 替换内存存储为文件流 #

使用multer.memoryStorage()会将文件全量加载到内存，大文件可能导致内存占用过高影响上传效率，可改用磁盘存储+文件流上传：

import * as fs from 'fs';

// 改用磁盘存储
const upload = multer({
  storage: multer.diskStorage({
    destination: './tmp',
    filename: (req, file, cb) => cb(null, `${Date.now()}-${file.originalname}`),
  }),
  fileFilter: options.fileFilter ?? ((req, file, cb) => cb(null, true)),
});

// 上传时使用文件流
await s3Client.send(
  new PutObjectCommand({
    Bucket: process.env.AWS_BUCKET_NAME,
    Key: key,
    Body: fs.createReadStream(file.path), // 替换buffer为文件流
    ContentType: file.mimetype,
  }),
);
// 上传完成后删除临时文件
fs.unlinkSync(file.path);

4. 检查存储桶权限与CORS配置 #

确保存储桶权限策略允许当前凭证执行PutObject操作，同时CORS配置允许服务域名访问：

• 存储桶权限策略示例：

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": { "AWS": "arn:aws:iam::你的账号ID:user/你的用户名" },
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::你的存储桶名/*"
    }
  ]
}

• CORS配置示例：

<CORSConfiguration>
  <CORSRule>
    <AllowedOrigin>https://你的服务域名</AllowedOrigin>
    <AllowedMethod>PUT</AllowedMethod>
    <AllowedHeader>*</AllowedHeader>
  </CORSRule>
</CORSConfiguration>

5. 启用SDK调试日志 #

通过环境变量或代码配置启用SDK详细日志，排查请求过程中的异常：

# 启动应用前设置
export AWS_SDK_LOG_LEVEL=debug

或代码中配置：

import { Logger } from "@aws-sdk/types";

const s3Client = new S3Client({
  region: process.env.AWS_REGION,
  credentials: {/* 凭证 */},
  logger: console as Logger,
});

6. 限制并发上传数量 #

同时上传多个文件可能超出客户端并发连接限制，可使用p-limit控制并发数：

import pLimit from 'p-limit';
const limit = pLimit(3); // 同时最多3个上传请求

const uploadPromises = allFiles.map(file => limit(async () => {
  const key = `${folderName}/${Date.now()}-${file.originalname}`;
  await s3Client.send(new PutObjectCommand({/* 配置项 */}));
  return {/* 文件信息 */};
}));

const uploadedFiles = await Promise.all(uploadPromises);

内容的提问来源于stack exchange，提问作者UAhmadSoft