Kafka S3 Sink Connector的value.subject.name.strategy配置无效求助
问题
尝试配置value.subject.name.strategy以替换默认的TopicNameStrategy,在worker.properties中添加了多种配置变体,但Kafka Sink Connector始终未识别,日志提示配置项为未知配置,默认仍使用TopicNameStrategy。
相关报错日志:
[2022-11-21 16:40:23,663] WARN The configuration 'value.converter.subject.name.strategy' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig:355) value.subject.name.strategy = class io.confluent.kafka.serializers.subject.TopicNameStrategy [2022-11-21 16:40:23,690] WARN The configuration 'converter.subject.name.strategy' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig:355) [2022-11-21 16:40:23,690] WARN The configuration 'value.subject.name.strategy' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig:355) [2022-11-21 16:40:23,690] WARN The configuration 'value.converter.subject.name.strategy' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig:355) [2022-11-21 16:40:23,719] WARN The configuration 'converter.subject.name.strategy' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig:355)
worker.properties中的相关配置:
grep -i "name.strategy" /plugins/worker.properties value.subject.name.strategy=io.confluent.kafka.serializers.subject.RecordNameStrategy value.converter.subject.name.strategy=io.confluent.kafka.serializers.subject.RecordNameStrategy consumer.value.subject.name.strategy=io.confluent.kafka.serializers.subject.RecordNameStrategy consumer.value.converter.subject.name.strategy=io.confluent.kafka.serializers.subject.RecordNameStrategy
正确配置方法
- 核心说明:
subject.name.strategy是Confluent Schema Registry专属Converter(如JsonSchemaConverter、AvroConverter)的配置项,不属于Kafka Consumer的配置范畴,因此不能添加consumer.前缀,也不能省略converter前缀。 - 全局配置(所有Connector生效):在
worker.properties中保留正确配置,同时指定对应的Converter:# 指定value使用的Confluent Schema Converter(根据实际序列化类型调整) value.converter=io.confluent.connect.json.JsonSchemaConverter # 配置Schema Registry地址 value.converter.schema.registry.url=http://your-schema-registry-host:8081 # 配置value的subject命名策略 value.converter.subject.name.strategy=io.confluent.kafka.serializers.subject.RecordNameStrategy - 单个Connector配置(仅目标Sink Connector生效):若无需全局生效,可在Connector的配置JSON中添加参数:
{ "name": "your-sink-connector", "config": { "value.converter": "io.confluent.connect.json.JsonSchemaConverter", "value.converter.schema.registry.url": "http://your-schema-registry-host:8081", "value.converter.subject.name.strategy": "io.confluent.kafka.serializers.subject.RecordNameStrategy", // 其他Connector配置项... } } - 清理无效配置:移除
worker.properties中以下无效项,避免日志报错:value.subject.name.strategyconsumer.value.subject.name.strategyconsumer.value.converter.subject.name.strategyconverter.subject.name.strategy
内容的提问来源于stack exchange,提问作者Jin Ma
相关产品推荐
相关产品推荐

