Spring Boot事务同步与回滚异常排查：ProducerFencedException问题

Hey there, let's break down your transaction sync/rollback issues and the occasional ProducerFencedException step by step. I’ve gone through all your configs and code, here are the key problems and their fixes:

1. ProducerFencedException: Duplicate Transaction IDs Across Instances #

The root cause here is your fixed transaction ID prefix sample-trans- in the producer factory. Kafka enforces that each transaction ID maps to exactly one producer instance. When you deploy multiple app instances using the same prefix, Kafka sees them as conflicting producers trying to reuse the same transaction ID, triggering the fencing mechanism and throwing this exception.

Fix: #

Add a unique instance identifier to your transaction ID prefix, like the server port or Spring Cloud instance ID:

@Value("${server.port}")
private String serverPort;

@Bean(value = "stringProducerFactory")
@Primary
public ProducerFactory<String, String> stringProducerFactory() {
    Map<String, Object> config = new ConcurrentHashMap<>();
    // ... keep your existing configs
    DefaultKafkaProducerFactory<String, String> defaultKafkaProducerFactory = new DefaultKafkaProducerFactory<>(config);
    // Append unique instance info to avoid conflicts
    defaultKafkaProducerFactory.setTransactionIdPrefix("sample-trans-" + applicationName + "-" + serverPort + "-");
    return defaultKafkaProducerFactory;
}

If you're using Spring Cloud, use ${spring.cloud.client.instance-id} instead of the port for a more robust unique ID.

2. Chained Transaction Manager: Wrong Order & Redundant Beans #

You’ve included both JpaTransactionManager and DataSourceTransactionManager in your ChainedKafkaTransactionManager, which creates unnecessary conflict—JPA’s transaction manager already wraps the datasource, so you don’t need both. Also, the order of managers in the chain is critical: database transactions should commit first, then Kafka transactions (and rollback in reverse order) to maintain consistency.

Fix: #

Remove the redundant DataSourceTransactionManager and reorder the chain:

@Bean(name = "chainedStringKafkaTransactionManager")
@Primary
public ChainedKafkaTransactionManager<String, String> chainedTransactionManager(JpaTransactionManager jpaTransactionManager) {
    // Database transaction first, Kafka transaction last
    return new ChainedKafkaTransactionManager<>(jpaTransactionManager, kafkaStringTransactionManager());
}

Also, explicitly specify this chained manager in your business method’s @Transactional annotation to ensure it’s used:

@Override
@Transactional(transactionManager = "chainedStringKafkaTransactionManager")
public void create(List<Employee> employees){
    // ... your existing code
}

3. Consumer Config: Auto-Commit vs Transaction Conflict #

Your consumer has ENABLE_AUTO_COMMIT_CONFIG=true, but you’re also using a transaction manager. These two settings clash—when using transactions, Kafka offset commits should be controlled by the transaction, not auto-committed in the background.

Fix: #

Disable auto-commit for your transactional consumers:

@Bean
public Map<String, Object> consumerConfigs() {
    Map<String, Object> props = new ConcurrentHashMap<>();
    // ... existing configs
    props.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, false); // Turn off auto-commit
    props.put(ConsumerConfig.ISOLATION_LEVEL_CONFIG, "read_committed");
    return props;
}

Keep the @Transactional on your consumer method since you want to tie offset commits to your processing logic.

4. Redundant Transaction Sync Configs #

You’ve set setNestedTransactionAllowed(true) and setTransactionSynchronization(SYNCHRONIZATION_ALWAYS) on the KafkaTransactionManager, but these are redundant (and potentially conflicting) when using a ChainedKafkaTransactionManager. Spring handles transaction synchronization automatically in the chain, so you can remove these:

@Bean(value = "stringKafkaTransactionManager")
public KafkaTransactionManager<String, String> kafkaStringTransactionManager() {
    KafkaTransactionManager<String, String> ktm = new KafkaTransactionManager<>(stringProducerFactory());
    // Remove these two lines
    // ktm.setNestedTransactionAllowed(true);
    // ktm.setTransactionSynchronization(AbstractPlatformTransactionManager.SYNCHRONIZATION_ALWAYS);
    return ktm;
}

5. Old Spring Boot Version: Known Transaction Bugs #

Spring Boot 2.1.4 (released in 2019) pairs with Spring Kafka 2.2.x, which has several known bugs around transaction synchronization and transaction ID handling. These could be contributing to your rollback issues.

Suggestion: #

Upgrade to a newer, stable Spring Boot version (like 2.3.12.RELEASE or later) if possible. This will pull in a more robust Spring Kafka version with fixes for many transaction-related issues. If you can’t upgrade, make sure you’re using the latest patch version of Spring Kafka 2.2.x.

6. Single Broker Kafka: Transaction Reliability Note #

While a single Kafka broker works for development, production environments should use at least 3 brokers to ensure transaction log durability and high availability. A single broker failure could lead to lost transaction metadata and inconsistent states.

内容的提问来源于stack exchange，提问作者Joyson Rego