使用AWS Lambda从S3获取文件超时，请求排查原因

问题现象 #

使用AWS Lambda从S3桶获取文件时持续超时，错误信息：

"errorMessage": "2025-05-16T14:37:13.093Z fdb6***4165 Task timed out after 30.03 seconds"

所用Lambda脚本 #

import boto3

def lambda_handler(event, context):
    bucket_name = "<myS3_bucket"
    file_key = "<path_to/file.csv>"

    s3 = boto3.resource('s3')
    for bucket in s3.buckets.all():
        print(bucket.name)

Terraform配置细节 #

IAM角色与策略 #

resource "aws_iam_role" "lambda_role" {
  name = "${var.lambda_name}_role"
  assume_role_policy = jsonencode({
    Statement = [
        {
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "lambda.amazonaws.com"
      }
    }]
  })
}

resource "aws_iam_role_policy_attachment" "S3_read_only" {
  role       = aws_iam_role.lambda_role.name
  policy_arn = "arn:aws:iam::aws:policy/AmazonS3FullAccess"
}
resource "aws_iam_role_policy_attachment" "lambda_logs" {
  role       = aws_iam_role.lambda_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}
resource "aws_iam_role_policy_attachment" "lambda_vpc_access" {
  role       = aws_iam_role.lambda_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole"
}

安全组配置 #

data "aws_vpc" "default" {
  default = true
}
resource "aws_security_group" "lambda_sg" {
  name        = "sg_${var.lambda_name}"
  description = "Allow all the ports needed for lambda"
  vpc_id      = data.aws_vpc.default.id

  # allow all inbound traffic
  ingress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  # allow all outbound traffic
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}
}

S3存储桶策略 #

resource "aws_s3_bucket_policy" "lambda_s3_access_policy" {
  bucket = "superset-dockerfiles"
  policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Effect = "Allow",
        Principal = {
          AWS = aws_iam_role.lambda_role.arn
        },
        Action = [
          "s3:GetObject",
          "s3:ListBucket"
        ],
        Resource = [
          "arn:aws:s3:::superset-dockerfiles",
          "arn:aws:s3:::superset-dockerfiles/*"
        ]
      }
    ]
  })
}

问题 #

请问我遗漏了什么配置？

排查与解决方向 #

1. VPC网络访问限制
   你的Lambda配置了VPC访问权限，但默认VPC中若无NAT网关或S3 VPC端点，Lambda无法访问公网S3服务：

   • 若无需Lambda在VPC内运行，直接移除VPC相关配置（包括安全组关联和lambda_vpc_access策略附件），Lambda会运行在AWS托管网络，默认可访问S3。
   • 必须保留VPC的话，需给VPC添加带公网IP的NAT网关，或创建S3的Gateway类型VPC端点，让Lambda通过内网访问S3。

2. 脚本效率问题
   s3.buckets.all()会遍历账号下所有S3桶，若桶数量多加上网络延迟，极易触发超时。建议直接指定目标桶操作，避免遍历：

   import boto3
   
   def lambda_handler(event, context):
       bucket_name = "superset-dockerfiles"
       file_key = "<path_to/file.csv>"
   
       s3 = boto3.resource('s3')
       bucket = s3.Bucket(bucket_name)
       # 获取目标文件内容示例
       obj = bucket.Object(file_key)
       content = obj.get()['Body'].read().decode('utf-8')
       print(content)
   

3. 超时时间配置
   当前Lambda超时为默认30秒，即便解决网络问题，遍历全量桶也可能超时。可在Terraform中为Lambda配置更长超时（最大15分钟），但优先解决网络和脚本问题更合理。

内容的提问来源于stack exchange，提问作者user15915737