Spring Cloud Stream Kafka Streams健康指示器仅显示1个线程的疑问
我在Spring Boot中使用Spring Cloud Stream binder运行Kafka Streams应用,该应用包含多个实例,每个实例配置了4个线程(num.stream.threads=4)。但访问Actuator健康端点(/actuator/health)时,每个实例仅显示1个线程的详情。请问这是预期行为还是我配置有误?
这是否意味着在REST端点中使用InteractiveQueryService时,只能查询该单个线程所属的状态存储,实例中剩余的3个线程无法被访问?
更新信息
健康端点kstream部分输出
kstream: { status: "UP", details: { streams.job-part-index-v1-1-20230405-1: { adminClientId: "streams.job-part-index-v1-1-20230405-1-7a89107e-6487-47ef-b1b6-d249546b53ab-admin", restoreConsumerClientId: "streams.job-part-index-v1-1-20230405-1-7a89107e-6487-47ef-b1b6-d249546b53ab-StreamThread-4-restore-consumer", threadState: "RUNNING", producerClientIds: [ "streams.job-part-index-v1-1-20230405-1-7a89107e-6487-47ef-b1b6-d249546b53ab-StreamThread-4-producer" ], standbyTasks: { }, activeTasks: { partitions: [ "partition=13, topic=integration.access-control-job-part-v1-0", "partition=13, topic=integration.job-part-v1-0", "partition=13, topic=streams.job-part-index-v1-1-20230405-1-COGROUPKSTREAM-AGGREGATE-STATE-STORE-0000000006-repartition" ], taskId: { topicGroupId: 1, partition: 13 } }, consumerClientId: "streams.job-part-index-v1-1-20230405-1-7a89107e-6487-47ef-b1b6-d249546b53ab-StreamThread-4-consumer", threadName: "streams.job-part-index-v1-1-20230405-1-7a89107e-6487-47ef-b1b6-d249546b53ab-StreamThread-4" } } }
运行日志(所有线程均正常运行)
{"timestamp":"2023-05-22 06:56:57.654","level":"INFO","thread":"streams.job-part-index-v1-1-20230405-1-7a89107e-6487-47ef-b1b6-d249546b53ab-StreamThread-1","logger":"org.apache.kafka.streams.processor.internals.StreamThread","message":"stream-thread [streams.job-part-index-v1-1-20230405-1-7a89107e-6487-47ef-b1b6-d249546b53ab-StreamThread-1] Processed 262 total records, ran 8 punctuators, and committed 29 total tasks since the last update","context":"default"} {"timestamp":"2023-05-22 06:57:29.502","level":"INFO","thread":"streams.job-part-index-v1-1-20230405-1-7a89107e-6487-47ef-b1b6-d249546b53ab-StreamThread-2","logger":"org.apache.kafka.streams.processor.internals.StreamThread","message":"stream-thread [streams.job-part-index-v1-1-20230405-1-7a89107e-6487-47ef-b1b6-d249546b53ab-StreamThread-2] Processed 280 total records, ran 4 punctuators, and committed 92 total tasks since the last update","context":"default"} {"timestamp":"2023-05-22 06:58:15.243","level":"INFO","thread":"streams.job-part-index-v1-1-20230405-1-7a89107e-6487-47ef-b1b6-d249546b53ab-StreamThread-3","logger":"org.apache.kafka.streams.processor.internals.StreamThread","message":"stream-thread [streams.job-part-index-v1-1-20230405-1-7a89107e-6487-47ef-b1b6-d249546b53ab-StreamThread-3] Processed 78 total records, ran 0 punctuators, and committed 67 total tasks since the last update","context":"default"} {"timestamp":"2023-05-22 06:58:15.628","level":"INFO","thread":"streams.job-part-index-v1-1-20230405-1-7a89107e-6487-47ef-b1b6-d249546b53ab-StreamThread-4","logger":"org.apache.kafka.streams.processor.internals.StreamThread","message":"stream-thread [streams.job-part-index-v1-1-20230405-1-7a89107e-6487-47ef-b1b6-d249546b53ab-StreamThread-4] Processed 81 total records, ran 0 punctuators, and committed 72 total tasks since the last update","context":"default"}
环境:Spring Boot 2.6.6、Spring Cloud Stream 3.2.1;输入分区数大于总线程数。
1. 健康端点仅显示单个线程是否为预期行为?
这是预期行为,并非配置错误。
Spring Cloud Stream的Kafka Streams binder健康指示器默认只展示单个StreamThread的详情,因为健康检查的核心目标是确认应用整体运行状态正常,而非枚举所有线程的细节。只要Kafka Streams拓扑处于运行状态,健康状态就会标记为UP,其他线程依然在正常处理任务(你的运行日志已验证这一点)。
如果需要查看所有线程的完整状态,可以通过自定义健康指示器实现,遍历Kafka Streams实例的所有StreamThread并收集状态信息。
2. InteractiveQueryService能否访问实例中所有线程的状态存储?
完全可以,不会受健康端点显示内容的影响。
InteractiveQueryService基于Kafka Streams的元数据定位状态存储,会自动感知当前实例内所有线程持有的状态分区。当查询状态时,它会根据数据对应的分区,自动路由到当前实例中负责该分区的线程,无论健康端点是否展示了该线程的信息。也就是说,实例内4个线程对应的状态存储都可以正常访问,不存在只能查询单个线程存储的限制。
内容的提问来源于stack exchange,提问作者user2732824

