@RetryTopic配置对Spring Boot Kafka应用性能的影响排查

架构老林

2026-7-8

Kafka非阻塞错误处理引入后性能下降10%的问题

现有一款已上线的Spring Boot v2.6.7应用，功能为从Kafka消费数据、转换后存储至数据库。在本地测试环境（Docker部署Kafka，单分区单消费者）下，处理100K条Kafka记录的耗时约为10分钟，三次测试数据如下：

first run: 10 min 20 sec.
second run: 10 min 05 sec.
third run: 10 min 36 sec.

为引入非阻塞错误处理，仅新增以下3个Bean，未修改其他配置：

@Bean
public KafkaTemplate<String, String> retryProducerKafkaTemplate() {
    return new KafkaTemplate<>(producerFactory());
}

@Bean
public ProducerFactory<String, String> producerFactory() {
    Map<String, Object> props = new HashMap<>();

    props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, kafkaProperties.getConsumer().getBootstrapServers());
    props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
    props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
    props.put(CommonClientConfigs.SECURITY_PROTOCOL_CONFIG, kafkaProperties.getConsumer().getSecurity().getProtocol());

    return new DefaultKafkaProducerFactory<>(props);
}

@Bean
public RetryTopicConfiguration retryTopicConfiguration() {
    return RetryTopicConfigurationBuilder
            .newInstance()
            .maxAttempts(2)
            .retryTopicSuffix("-retry")
            .doNotAutoCreateRetryTopics()
            .dltSuffix("-dlt")
            .includeTopics(consumerTopics)
            .exponentialBackoff(1000, 2, 10000)
            .retryOn(List.of(DataAccessException.class))
            .dltHandlerMethod("eventsListener", "dlt")
            .create(retryProducerKafkaTemplate());
}

再次测试处理100K条记录，耗时变为约11分钟，三次测试数据如下：