Kafka Avro主题访问报错:IllegalStateException重复键enum问题
Avro Schema访问Kafka主题时出现Duplicate key enum错误排查
使用Avro Schema访问Kafka主题sims.sim_events_v2时触发以下错误,但主题消息可正常解码,无报错:
java.lang.IllegalStateException: Duplicate key enum (attempted merging values com.provectus.kafka.ui.util.jsonschema.SimpleFieldSchema@11e16b91 and com.provectus.kafka.ui.util.jsonschema.SimpleFieldSchema@22fb626a) at java.base/java.util.stream.Collectors.duplicateKeyException(Collectors.java:135) Suppressed: The stacktrace has been enhanced by Reactor, refer to additional information below: Error has been observed at the following site(s): *__checkpoint ⇢ com.provectus.kafka.ui.config.CorsGlobalConfiguration$$Lambda$1007/0x00000008016952f0 [DefaultWebFilterChain] *__checkpoint ⇢ com.provectus.kafka.ui.config.CustomWebFilter [DefaultWebFilterChain] *__checkpoint ⇢ com.provectus.kafka.ui.config.ReadOnlyModeFilter [DefaultWebFilterChain] *__checkpoint ⇢ AuthorizationWebFilter [DefaultWebFilterChain] *__checkpoint ⇢ ExceptionTranslationWebFilter [DefaultWebFilterChain] *__checkpoint ⇢ LogoutWebFilter [DefaultWebFilterChain] *__checkpoint ⇢ ServerRequestCacheWebFilter [DefaultWebFilterChain] *__checkpoint ⇢ SecurityContextServerWebExchangeWebFilter [DefaultWebFilterChain] *__checkpoint ⇢ ReactorContextWebFilter [DefaultWebFilterChain] *__checkpoint ⇢ HttpHeaderWriterWebFilter [DefaultWebFilterChain] *__checkpoint ⇢ ServerWebExchangeReactorContextWebFilter [DefaultWebFilterChain] *__checkpoint ⇢ org.springframework.security.web.server.WebFilterChainProxy [DefaultWebFilterChain] *__checkpoint ⇢ org.springframework.web.filter.reactive.ServerHttpObservationFilter [DefaultWebFilterChain] *__checkpoint ⇢ HTTP GET "/api/clusters/local-kafka/topic/sims.sim_events_v2/serdes?use=SERIALIZE" [ExceptionHandlingWebHandler]Original Stack Trace: at java.base/java.util.stream.Collectors.duplicateKeyException(Collectors.java:135) at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:182) at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179) at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) at ...
当前使用的Avro Schema:
{ "namespace": "sims.sim_events_v2", "type": "record", "name": "SIMEvent", "fields": [ { "name": "event_id", "type": "string" } ] }
Schema Subject:
sims.sim_events_v2-value
队列名称:sims.sim_events_v2
错误分析与排查方向
这个错误的核心是Kafka UI(从报错路径com.provectus.kafka.ui可判断)在将Avro Schema转换为JSON Schema时,出现了重复的enum键冲突。消息能正常解码说明Avro Schema本身在序列化/反序列化逻辑上是有效的,问题出在Kafka UI的Schema转换环节。
- 检查Schema Registry中
sims.sim_events_v2-valueSubject下的所有Schema版本,确认是否存在不同版本的Schema包含同名enum定义,或enum元数据冲突。即使当前使用的Schema没有enum,旧版本Schema可能残留相关定义,导致Kafka UI合并元数据时出错。 - 查看你使用的Kafka UI版本,该错误可能是特定版本的已知bug。建议升级到最新稳定版,很多类似的Schema转换问题在新版本中已修复。
- 确认是否同时配置了key的Schema(当前仅提供了value的Subject)。如果key的Schema存在enum定义冲突,也可能触发这个错误。
- 若确认当前Avro Schema是正确的,可以尝试重新注册该Schema到
sims.sim_events_v2-valueSubject,覆盖旧版本,或者清理Schema Registry中该Subject下的无效历史版本。
内容的提问来源于stack exchange,提问作者Radosław Z
相关产品推荐
相关产品推荐

