You need to enable JavaScript to run this app.
优惠活动
大模型
产品
解决方案
定价
更多

Kafka S3 Connect报错‘Value is not Struct type’求助

Confluent S3 Sink Connector报错:Value is not Struct type(FieldPartitioner使用问题)

最近我在使用Confluent Open Source v4.1部署S3 Sink Connector时遇到了棘手问题——之前运行正常的配置,现在抛出了分区编码错误,完全无法处理消息。

我的配置与操作步骤

1. 加载S3 Sink Connector的命令

我用confluent load命令加载了Connector,配置参数如下:

confluent load s3-sink { "name": "s3-sink", "config": { "connector.class": "io.confluent.connect.s3.S3SinkConnector", "tasks.max": "1", "topics": "s3_topic", "s3.region": "us-east-1", "s3.bucket.name": "some_bucket", "s3.part.size": "5242880", "flush.size": "1", "storage.class": "io.confluent.connect.s3.storage.S3Storage", "format.class": "io.confluent.connect.s3.format.json.JsonFormat", "schema.generator.class": "io.confluent.connect.storage.hive.schema.DefaultSchemaGenerator", "partitioner.class": "io.confluent.connect.storage.partitioner.FieldPartitioner", "schema.compatibility": "NONE", "partition.field.name": "f1", "key.converter": "org.apache.kafka.connect.json.JsonConverter", "value.converter": "org.apache.kafka.connect.json.JsonConverter", "key.converter.schemas.enable": "false", "value.converter.schemas.enable": "false", "name": "s3-sink" }, "tasks": [ { "connector": "s3-sink", "task": 0 } ], "type": null }

2. 发送测试消息

通过kafka-console-producer发送了一条JSON格式的消息:

{"f1":"partition","data":"some data"}

遇到的错误日志

结果在Connect的日志里看到了这样的错误:

[2018-05-16 16:32:05,150] ERROR Value is not Struct type. (io.confluent.connect.storage.partitioner.FieldPartitioner:67)
[2018-05-16 16:32:05,150] ERROR WorkerSinkTask{id=s3-sink-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted. (org.apache.kafka.connect.runtime.WorkerSinkTask:515)
io.confluent.connect.storage.errors.PartitionException: Error encoding partition.

这个配置在之前的版本里是完全正常的,升级到v4.1之后就出问题了。有没有大佬能帮忙分析一下问题出在哪,怎么解决?


内容的提问来源于stack exchange,提问作者Vova l

相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级,最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分), 限时 9.9 元起

最近更新时间:2026.05.28 04:04:19