咨询Terraform中Firehose S3目的地启用换行分隔符的对应参数
启用Firehose S3目的地换行分隔符的Terraform参数
在你描述的场景(数据源为对接DynamoDB的Kinesis流、S3作为目的地、未配置动态分区与Lambda)中,启用「换行分隔符」对应的Terraform参数有两种常用配置方式:
方式一:通过JSON序列化器直接启用
在aws_kinesis_firehose_delivery_stream资源的s3_configuration.output_format_configuration.serializer块内,设置newline_delimited = true,同时指定格式为JSON(匹配DynamoDB流的默认数据格式)。
示例配置片段:
resource "aws_kinesis_firehose_delivery_stream" "example_stream" { name = "dynamodb-to-s3-firehose" destination = "s3" kinesis_stream_source_configuration { kinesis_stream_arn = aws_kinesis_stream.dynamodb_stream.arn role_arn = aws_iam_role.firehose_exec_role.arn } s3_configuration { role_arn = aws_iam_role.firehose_exec_role.arn bucket_arn = aws_s3_bucket.firehose_destination.arn output_format_configuration { serializer { format = "JSON" newline_delimited = true # 启用换行分隔符 } } } }
方式二:通过内置处理器添加换行符
如果需要自定义分隔符(此处指定为换行符),可以启用Firehose内置的AppendDelimiterToRecord处理器:
resource "aws_kinesis_firehose_delivery_stream" "example_stream" { name = "dynamodb-to-s3-firehose" destination = "s3" kinesis_stream_source_configuration { kinesis_stream_arn = aws_kinesis_stream.dynamodb_stream.arn role_arn = aws_iam_role.firehose_exec_role.arn } s3_configuration { role_arn = aws_iam_role.firehose_exec_role.arn bucket_arn = aws_s3_bucket.firehose_destination.arn } processing_configuration { enabled = true processors { type = "AppendDelimiterToRecord" parameters { parameter_name = "Delimiter" parameter_value = "\n" } } } }
配图:
内容的提问来源于stack exchange,提问作者hari krishnan
相关产品推荐
相关产品推荐

