在AWS CDK栈中配置MSK凭证遇部署错误求助

报错信息 #

10:55:25 AM | CREATE_FAILED        | AWS::Lambda::EventSourceMapping | QFDSKafkaEventSour...iltynotifyEFE73996
Resource handler returned message: "Invalid request provided: The secret provided in 'sourceAccessConfigurations' is not associated with cluster some-valid-an. Please provide a secret associated with the cluster. (Service: Lambda, Status Code: 400, Request ID: some-uuid )" (RequestToken: some-uuid, HandlerErrorCode: InvalidRequest)

问题背景 #

通过CDK搭建栈，已实现Lambda、Lambda层及基础触发器部署，但生产环境MSK集群需SCRAM凭证。Secrets Manager中已创建路径为integrations/msk/creds的scram-sha-512密钥，无法正确关联MSK集群，同时不清楚Secret()构造函数第二个参数的用途，导致部署失败。

现有CDK代码 #

#!/usr/bin/env python3
from aws_cdk import (
    aws_lambda as lambda_,
    App, Duration, Stack
)
from aws_cdk.aws_lambda_event_sources import ManagedKafkaEventSource
from aws_cdk.aws_secretsmanager import Secret

class ExternalRestEndpoint(Stack):

    def __init__(self, app: App, id: str) -> None:
        super().__init__(app, id)

        secret = Secret(self, "Secret", secret_name="integrations/msk/creds")

        msk_arn = "some valid and confirmed arn"

        # Lambda layer.
        lambdaLayer = lambda_.LayerVersion(self, 'lambda-layer',
            code = lambda_.AssetCode('utils/lambda-deployment-packages/lambda-layer.zip'),
            compatible_runtimes = [lambda_.Runtime.PYTHON_3_7],
        )  

        # Source for the lambda.
        with open("src/path/to/sourcefile.py", encoding="utf8") as fp:
            mysource_code = fp.read()

        # Config for it.
        lambdaFn = lambda_.Function(
            self, "QFDS",
            code=lambda_.InlineCode(mysource_code),
            handler="lambda_handler",
            timeout=Duration.seconds(300),
            runtime=lambda_.Runtime.PYTHON_3_7,
            layers=[lambdaLayer],
        )

        # Set up the event (managed Kafka). 
        lambdaFn.add_event_source(ManagedKafkaEventSource(
            cluster_arn=prototype_mks,
            topic="foreign.endpoint.availabilty.notify", 
            secret=secret,
            batch_size=100,                # default
            starting_position=lambda_.StartingPosition.TRIM_HORIZON
        ))

1. 关联Secret与MSK集群 #

创建Secret后必须将其关联到目标MSK集群，这是报错的核心原因：

• 控制台操作：进入MSK集群详情页 → 「安全」标签 → 「SCRAM凭证」→ 「添加凭证」，选择integrations/msk/creds密钥。
• AWS CLI操作：执行以下命令直接关联：

  aws kafka associate-scram-secret --cluster-arn <你的MSK集群ARN> --secret-arns <你的Secret ARN>
  

• 验证关联：执行命令确认密钥已关联：

  aws kafka list-scram-secrets --cluster-arn <你的MSK集群ARN>
  

2. 修正CDK代码中的笔误 #

代码中存在变量引用错误：

• 将ManagedKafkaEventSource中的cluster_arn=prototype_mks替换为cluster_arn=msk_arn，因为prototype_mks未定义，实际已声明的MSK集群ARN变量是msk_arn。

3. 理解Secret()构造函数参数 #

Secret(self, "Secret", secret_name="integrations/msk/creds")中：

• 第二个参数"Secret"是CDK栈内的资源逻辑ID，仅用于CDK内部标识该资源，与Secrets Manager中的密钥名称无关。
• secret_name参数才是指定Secrets Manager中密钥的实际路径/名称。

4. 确认Lambda权限（可选） #

CDK的ManagedKafkaEventSource会自动为Lambda执行角色添加访问该Secret的secretsmanager:GetSecretValue权限，若存在自定义权限策略，需确保该权限未被移除。

内容的提问来源于stack exchange，提问作者John O