配置S3新文件触发Lambda遇验证错误，寻求解决方法

核心问题分析 #

你遇到的Unable to validate the following destination configurations报错，本质是S3缺少向目标资源（SNS/SQS）发送事件的权限，或者目标资源的访问策略未开放给S3。另外，对于你的场景，绝大多数情况不需要SNS/SQS中间件，直接让S3触发Lambda是最简洁高效的方案。

方案一：直接S3触发Lambda（推荐） #

不需要额外的SNS/SQS，直接配置S3对象创建事件触发Lambda，适合单文件处理的常规场景。

修正后的SAM模板 #

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: S3 -> Lambda -> Redshift 数据上传流程

Globals:
  Function:
    Timeout: 600
    MemorySize: 128
    Tracing: Active

Parameters:
  S3BucketName:
    Type: String
    Description: 要监控的S3桶名称
  RedshiftKmsKeyId:
    Type: String
    Description: Redshift关联的KMS密钥ARN
  RedshiftDataApiArn:
    Type: String
    Description: Redshift Data API资源ARN
  SecretsManagerSecretArn:
    Type: String
    Description: 存储Redshift凭证的Secrets Manager ARN

Resources:
  RSUploadFunction:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: RSUploadFunction
      CodeUri: upload_to_rs/
      Handler: app.lambda_handler
      Runtime: python3.9
      PackageType: Zip
      Architectures:
      - x86_64
      Policies:
        - Statement:
          - Sid: KMSGenerateDataKey
            Effect: Allow
            Action:
            - kms:GenerateDataKey
            Resource: !Ref RedshiftKmsKeyId
        - Statement:
          - Sid: ExecuteRedshiftStatement
            Effect: Allow
            Action:
            - redshift-data:ExecuteStatement
            Resource: !Ref RedshiftDataApiArn
        - Statement:
          - Sid: DescribeRedshiftStatement
            Effect: Allow
            Action:
            - redshift-data:DescribeStatement
            Resource: '*'
        - KMSDecryptPolicy:
            KeyId: !Ref RedshiftKmsKeyId
        - KMSEncryptPolicy:
            KeyId: !Ref RedshiftKmsKeyId
        - S3CrudPolicy:
            BucketName: !Ref S3BucketName
        - AWSSecretsManagerGetSecretValuePolicy:
            SecretArn: !Ref SecretsManagerSecretArn
      # 直接配置S3事件触发
      Events:
        S3FileUpload:
          Type: S3
          Properties:
            Bucket: !Ref S3BucketName
            Events: s3:ObjectCreated:*
            # 可选：过滤特定后缀/前缀的文件，避免无关触发
            Filter:
              S3Key:
                Rules:
                  - Name: suffix
                    Value: .csv

Outputs:
  RSUploadFunctionArn:
    Description: Lambda函数ARN
    Value: !GetAtt RSUploadFunction.Arn

部署与验证 #

1. 用sam deploy --guided命令部署模板，按提示填写参数。
2. 部署完成后，S3会自动获得触发Lambda的权限，无需手动在S3控制台配置事件（SAM已帮你完成）。
3. 上传测试文件到S3桶，查看Lambda日志确认是否触发。

方案二：S3 -> SNS -> SQS -> Lambda（特殊场景） #

如果需要批量处理、重试机制或多消费者解耦，可采用此架构。以下是修正权限后的模板：

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: S3 -> SNS -> SQS -> Lambda 数据上传流程

Globals:
  Function:
    Timeout: 600
    MemorySize: 128
    Tracing: Active

Parameters:
  S3BucketName:
    Type: String
    Description: 要监控的S3桶名称
  RedshiftKmsKeyId:
    Type: String
    Description: Redshift关联的KMS密钥ARN
  RedshiftDataApiArn:
    Type: String
    Description: Redshift Data API资源ARN
  SecretsManagerSecretArn:
    Type: String
    Description: 存储Redshift凭证的Secrets Manager ARN

Resources:
  RSUploadFunction:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: RSUploadFunction
      CodeUri: upload_to_rs/
      Handler: app.lambda_handler
      Runtime: python3.9
      PackageType: Zip
      Architectures:
      - x86_64
      Policies:
        - Statement:
          - Sid: KMSGenerateDataKey
            Effect: Allow
            Action:
            - kms:GenerateDataKey
            Resource: !Ref RedshiftKmsKeyId
        - Statement:
          - Sid: ExecuteRedshiftStatement
            Effect: Allow
            Action:
            - redshift-data:ExecuteStatement
            Resource: !Ref RedshiftDataApiArn
        - Statement:
          - Sid: DescribeRedshiftStatement
            Effect: Allow
            Action:
            - redshift-data:DescribeStatement
            Resource: '*'
        - KMSDecryptPolicy:
            KeyId: !Ref RedshiftKmsKeyId
        - KMSEncryptPolicy:
            KeyId: !Ref RedshiftKmsKeyId
        - S3CrudPolicy:
            BucketName: !Ref S3BucketName
        - AWSSecretsManagerGetSecretValuePolicy:
            SecretArn: !Ref SecretsManagerSecretArn
        # 允许Lambda读取SQS消息
        - SQSPollerPolicy:
            QueueName: !GetAtt SQSQueue.QueueName
      # 配置SQS作为Lambda触发源
      Events:
        SQSTrigger:
          Type: SQS
          Properties:
            Queue: !GetAtt SQSQueue.Arn
            BatchSize: 10 # 批量处理消息数，按需调整

  SNSTopic:
    Type: AWS::SNS::Topic
    Properties:
      DisplayName: S3FileUploadTopic
      TopicName: S3FileUploadTopic
      # SNS订阅SQS
      Subscription:
        - Endpoint: !GetAtt SQSQueue.Arn
          Protocol: sqs

  # 配置SNS权限：允许S3发送消息到Topic
  SNSTopicPolicy:
    Type: AWS::SNS::TopicPolicy
    Properties:
      Topics:
        - !Ref SNSTopic
      PolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: s3.amazonaws.com
            Action: sns:Publish
            Resource: !Ref SNSTopic
            Condition:
              ArnEquals:
                aws:SourceArn: !Sub 'arn:aws:s3:::${S3BucketName}'

  SQSQueue:
    Type: AWS::SQS::Queue
    Properties:
      VisibilityTimeout: 600
      QueueName: S3FileUploadQueue

  # 配置SQS权限：允许SNS发送消息到Queue
  SQSQueuePolicy:
    Type: AWS::SQS::QueuePolicy
    Properties:
      PolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: sns.amazonaws.com
            Action: sqs:SendMessage
            Resource: !GetAtt SQSQueue.Arn
            Condition:
              ArnEquals:
                aws:SourceArn: !Ref SNSTopic
      Queues: 
        - !Ref SQSQueue

Outputs:
  RSUploadFunctionArn:
    Description: Lambda函数ARN
    Value: !GetAtt RSUploadFunction.Arn
  SNSTopicArn:
    Description: SNS主题ARN
    Value: !Ref SNSTopic
  SQSQueueArn:
    Description: SQS队列ARN
    Value: !GetAtt SQSQueue.Arn

解决S3事件配置报错的关键 #

之前的报错是因为缺少S3向SNS/SQS发送事件的权限：

• 若目标是SNS：必须添加SNSTopicPolicy，允许S3服务调用sns:Publish。
• 若目标是SQS：必须修改Queue Policy，允许S3服务调用sqs:SendMessage（而非仅允许SNS）。

内容的提问来源于stack exchange，提问作者Raksha