CDK技术问询：如何在CloudFormationInit中使用OpenSearch主用户密码

问题描述 #

在使用AWS CDK创建OpenSearch L2构造并开启细粒度访问控制后，Secrets Manager会自动生成包含主用户凭据的密钥。需要在后续的CloudFormationInit中，将该凭据填入Logstash配置文件，但无法确定自动生成的Secret ID/ARN，且SecretValue没有直接的secretId属性。相关代码示例如下：

from aws_cdk import aws_ec2 as ec2
from aws_cdk import aws_iam as iam
from aws_cdk import aws_opensearchservice as opensearch
from aws_cdk import aws_s3 as s3


class OpensearchStack(Stack):
    def __init__(
        self,
        scope: Construct,
        construct_id: str,
        **kwargs,
    ) -> None:
        super().__init__(scope, construct_id, **kwargs)

        vpc = ec2.Vpc(self, "generatorVpc", max_azs=2)
        bucket = s3.Bucket(self, "My Bucket")
        domain = opensearch.Domain(self,"OpensearchDomain",
            version=opensearch.EngineVersion.OPENSEARCH_1_3,
            vpc=vpc,
            fine_grained_access_control=opensearch.AdvancedSecurityOptions(
                master_user_name="osadmin",
            ),
        )
        instance = ec2.Instance(self, "Instance",
            vpc=vpc,
            instance_type=ec2.InstanceType.of(
                instance_class=ec2.InstanceClass.M5,
                instance_size=ec2.InstanceSize.LARGE,
            ),
            machine_image=ec2.MachineImage.latest_amazon_linux(
                generation=ec2.AmazonLinuxGeneration.AMAZON_LINUX_2,
            ),
            init=ec2.CloudFormationInit.from_elements(
                ec2.InitFile.from_string(
                            file_name="/home/ec2-user/logstash-8.4.0/config/my_conf.conf",
                            owner="ec2-user",
                            mode="00755",
                            content=f"""input {{
    s3 {{
        bucket => "{bucket.bucket_name}"
        region => "{self.region}"
    }}
}}
output {{
    opensearch {{
        hosts => ["{domain.domain_endpoint}:443"]
        user => "{domain.master_user_password.secrets_manager("What secret id do I put here?", json_field="username")}"
        password => "{domain.master_user_password.secrets_manager("What secret id do I put here?", json_field="password")}"
        ecs_compatibility => disabled
    }}
}}
""",
                )
            )
        )

解决方案 #

1. 获取自动生成的Secret ARN #

当你通过AdvancedSecurityOptions指定master_user_name但不提供master_user_password时，CDK会自动创建Secrets Manager密钥，该密钥的ARN可通过OpenSearch Domain的master_user_secret.secret_arn属性直接获取。

2. 为EC2实例添加读取Secret的权限 #

EC2实例需要拥有读取该Secret的IAM权限，否则初始化过程中无法解析凭据：

# 给EC2实例的角色添加读取Secret的权限
instance.role.add_to_policy(iam.PolicyStatement(
    actions=["secretsmanager:GetSecretValue"],
    resources=[domain.master_user_secret.secret_arn]
))

3. 在Logstash配置中动态引用Secret值 #

使用CDK的SecretValue.secrets_manager()方法生成CloudFormation动态引用字符串，直接嵌入到配置文件中：

# 生成用户名和密码的动态引用
os_user = str(opensearch.SecretValue.secrets_manager(
    domain.master_user_secret.secret_arn,
    json_field="username"
))
os_password = str(opensearch.SecretValue.secrets_manager(
    domain.master_user_secret.secret_arn,
    json_field="password"
))

# 初始化文件内容
init_file = ec2.InitFile.from_string(
    file_name="/home/ec2-user/logstash-8.4.0/config/my_conf.conf",
    owner="ec2-user",
    mode="00755",
    content=f"""input {{
    s3 {{
        bucket => "{bucket.bucket_name}"
        region => "{self.region}"
    }}
}}
output {{
    opensearch {{
        hosts => ["{domain.domain_endpoint}:443"]
        user => "{os_user}"
        password => "{os_password}"
        ecs_compatibility => disabled
    }}
}}
"""
)

# 实例初始化
instance = ec2.Instance(self, "Instance",
    vpc=vpc,
    instance_type=ec2.InstanceType.of(
        instance_class=ec2.InstanceClass.M5,
        instance_size=ec2.InstanceSize.LARGE,
    ),
    machine_image=ec2.MachineImage.latest_amazon_linux(
        generation=ec2.AmazonLinuxGeneration.AMAZON_LINUX_2,
    ),
    init=ec2.CloudFormationInit.from_elements(init_file)
)

原理说明 #

SecretValue.secrets_manager()会生成类似{{resolve:secretsmanager:arn:aws:secretsmanager:region:account-id:secret:secret-name:SecretString:username}}的字符串，CloudFormation在创建EC2实例时会自动解析该引用，将实际的用户名和密码填入配置文件中。

内容的提问来源于stack exchange，提问作者maafk