Provectus Kafka UI无法加载主题详情问题求助

问题描述 #

我在服务器部署了kafka-UI容器，启动命令如下：

docker run -p 8080:8080 --name kafka-ui -e KAFKA_CLUSTERS_0_NAME=local -e KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=XX.XX.XX.XX:9093 -d provectuslabs/kafka-ui:latest

现象 #

Web控制台能加载Broker详情，但无法加载「主题」和「消费者」详情。

容器日志报错 #

2023-05-16 14:36:44,326 INFO  [kafka-admin-client-thread | kafka-ui-admin-1684244881-1] o.a.k.c.NetworkClient: [AdminClient clientId=kafka-ui-admin-1684244881-1] Disconnecting from node 261 due to socket connection setup timeout. The timeout value is 32528 ms.
2023-05-16 14:36:44,846 DEBUG [parallel-33] c.p.k.u.s.ClustersStatisticsScheduler: Metrics updated for cluster: local
2023-05-16 14:36:44,848 DEBUG [parallel-34] c.p.k.u.s.ClustersStatisticsScheduler: Start getting metrics for kafkaCluster: local
2023-05-16 14:36:49,053 ERROR [parallel-35] o.s.b.a.w.r.e.AbstractErrorWebExceptionHandler: [b09cf81d-30]  500 Server Error for HTTP GET "/api/clusters/local/topics?page=1&perPage=25&showInternal=false"
org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the call. Call: listOffsets on broker 261
    Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Error has been observed at the following site(s):
    *__checkpoint ⇢ Handler com.provectus.kafka.ui.controller.TopicsController#getTopics(String, Integer, Integer, Boolean, String, TopicColumnsToSortDTO, SortOrderDTO, ServerWebExchange) [DispatcherHandler]
    *__checkpoint ⇢ com.provectus.kafka.ui.config.CustomWebFilter [DefaultWebFilterChain]
    *__checkpoint ⇢ com.provectus.kafka.ui.config.ReadOnlyModeFilter [DefaultWebFilterChain]
    *__checkpoint ⇢ AuthorizationWebFilter [DefaultWebFilterChain]
    *__checkpoint ⇢ ExceptionTranslationWebFilter [DefaultWebFilterChain]
    *__checkpoint ⇢ LogoutWebFilter [DefaultWebFilterChain]
    *__checkpoint ⇢ ServerRequestCacheWebFilter [DefaultWebFilterChain]
    *__checkpoint ⇢ SecurityContextServerWebExchangeWebFilter [DefaultWebFilterChain]
    *__checkpoint ⇢ ReactorContextWebFilter [DefaultWebFilterChain]
    *__checkpoint ⇢ HttpHeaderWriterWebFilter [DefaultWebFilterChain]
    *__checkpoint ⇢ ServerWebExchangeReactorContextWebFilter [DefaultWebFilterChain]
    *__checkpoint ⇢ org.springframework.security.web.server.WebFilterChainProxy [DefaultWebFilterChain]
    *__checkpoint ⇢ org.springframework.web.filter.reactive.ServerHttpObservationFilter [DefaultWebFilterChain]
    *__checkpoint ⇢ HTTP GET "/api/clusters/local/topics?page=1&perPage=25&showInternal=false" [ExceptionHandlingWebHandler]

已验证信息 #

• 用相同的bootstrap端点手动执行Kafka主题列表命令可正常获取所有主题，Kafka集群运行正常
• 已尝试设置KAFKA_UI_JVM_OPTS="-Xmx1g -Dserver.servlet.session.timeout=600"增加容器资源，问题未解决

解决方案 #

1. 检查Kafka Broker的advertised.listeners配置 #

Kafka UI能连接bootstrap服务器但无法访问具体Broker节点，大概率是Broker的对外地址配置错误：

• 确认Kafka Broker的advertised.listeners设置为Kafka UI容器可访问的公网IP/域名，而非内部IP（如容器内网IP、localhost）
• 若Kafka部署在容器或云环境中，advertised.listeners必须指向外部可访问的地址，否则Kafka UI拿到Broker的内部地址后无法建立连接

2. 调整Kafka UI的AdminClient超时参数 #

日志显示listOffsets调用超时，可通过环境变量延长超时时间，修改启动命令如下：

docker run -p 8080:8080 --name kafka-ui \
-e KAFKA_CLUSTERS_0_NAME=local \
-e KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=XX.XX.XX.XX:9093 \
-e KAFKA_CLUSTERS_0_PROPERTIES_ADMIN_REQUEST_TIMEOUT_MS=60000 \
-e KAFKA_CLUSTERS_0_PROPERTIES_REQUEST_TIMEOUT_MS=60000 \
-e KAFKA_CLUSTERS_0_PROPERTIES_CONNECTIONS_MAX_IDLE_MS=300000 \
-d provectuslabs/kafka-ui:latest

3. 验证Kafka UI容器到所有Broker节点的网络连通性 #

进入Kafka UI容器，测试到报错Broker（节点261）的连通性：

# 测试IP可达性
docker exec -it kafka-ui ping <Broker-261-IP>
# 测试端口连通性
docker exec -it kafka-ui telnet <Broker-261-IP> <Broker监听端口>

若无法连通，需检查防火墙、安全组规则，确保Kafka UI容器能访问所有Kafka Broker的监听端口（默认9092/9093）

4. 确认Kafka UI与Kafka集群版本兼容性 #

部分新版本Kafka UI可能与旧版Kafka集群存在兼容性问题，可尝试指定稳定版本启动：

docker run -p 8080:8080 --name kafka-ui -e KAFKA_CLUSTERS_0_NAME=local -e KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=XX.XX.XX.XX:9093 -d provectuslabs/kafka-ui:v0.7.2

内容的提问来源于stack exchange，提问作者Danuka