如何配置EventBridge Sink Connector将Kafka消息内容扁平化至detail顶层？

我正在使用Amazon EventBridge Sink Connector for Kafka Connect将Kafka主题中的消息发送至EventBridge。Kafka中的源消息格式如下：

{"headers": {"country": "US","channel": "finance dept","processType": "Update-usecases"},"payload": {"date": "01/01/2025","address": "JOE ST APT 0000","customerId": "00000","accountNumber": "00000000"}}

但消息到达EventBridge后，被包裹在detail对象的value字段中：

"detail": {
  "topic": "test_topic",
  "partition": "1",
  "offset": "28",
  "timestamp": "1749142570349",
  "timestampType": "CreateTime",
  "key": "18",
  "value": {"headers": { ... },"payload": { ... }}
}

我的期望格式是headers和payload直接作为detail的顶层字段：

"detail": {
  "topic": "test_topic",
  "partition": "1",
  "offset": "28",
  "timestamp": "1749142570349",
  "timestampType": "CreateTime",
  "key": "18",
  "headers": { ... },
  "payload": { ... }
}

当前连接器配置如下：

{
  "connector.class": "software.amazon.event.kafkaconnector.EventBridgeSinkConnector",
  "topics": "test_topic",
  "value.converter": "org.apache.kafka.connect.json.JsonConverter",
  "value.converter.schemas.enable": "false",
  "aws.eventbridge.source": "gravity",
  "aws.eventbridge.detail.types": "gravity.Undate-usecases",
  ...
}

请问是否可通过调整连接器配置，将Kafka消息内容置于detail对象顶层而非嵌套在value字段下？希望得到相关配置调整建议。

可以通过添加以下两个配置参数实现需求：

• aws.eventbridge.detail.include.value：设置为false，关闭默认将Kafka消息体嵌套在value字段的行为
• aws.eventbridge.detail.fields：指定要从Kafka消息中提取并直接放入detail顶层的字段，这里设置为headers,payload

修改后的完整连接器配置示例：

{
  "connector.class": "software.amazon.event.kafkaconnector.EventBridgeSinkConnector",
  "topics": "test_topic",
  "value.converter": "org.apache.kafka.connect.json.JsonConverter",
  "value.converter.schemas.enable": "false",
  "aws.eventbridge.source": "gravity",
  "aws.eventbridge.detail.types": "gravity.Update-usecases",
  "aws.eventbridge.detail.include.value": "false",
  "aws.eventbridge.detail.fields": "headers,payload"
}

配置说明： #

• aws.eventbridge.detail.include.value=false：禁用连接器默认把消息体封装到value字段的逻辑
• aws.eventbridge.detail.fields=headers,payload：明确指定要将Kafka消息里的headers和payload字段直接映射到EventBridge事件的detail顶层，与topic、partition等元数据平级

配置生效后，EventBridge收到的消息detail结构就会符合你的期望格式。

内容的提问来源于stack exchange，提问作者Ekele James