Great question — you’re right that the global transaction-id-prefix is the first thing most developers encounter, but Spring Kafka (paired with Spring Cloud Stream) absolutely supports per-binding transaction configuration—you don’t have to use a one-size-fits-all global setting. Here’s how to implement this for your specific use case:

1. Create Separate ProducerFactory Beans #

First, define two distinct ProducerFactory beans in your configuration: one for transactional use cases (retry/DLT message transfers) and one for non-transactional use cases (outbox message sending).

import org.apache.kafka.clients.producer.ProducerConfig;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.kafka.core.DefaultKafkaProducerFactory;
import org.springframework.kafka.core.ProducerFactory;
import java.util.Map;

@Configuration
public class KafkaProducerConfig {

    // Non-transactional ProducerFactory for outbox messages
    @Bean("nonTransactionalProducerFactory")
    public ProducerFactory<String, Object> nonTransactionalProducerFactory(Map<String, Object> kafkaProducerConfigs) {
        // Remove transaction-related configs or explicitly disable
        kafkaProducerConfigs.remove(ProducerConfig.TRANSACTION_ID_PREFIX_CONFIG);
        return new DefaultKafkaProducerFactory<>(kafkaProducerConfigs);
    }

    // Transactional ProducerFactory for retry/DLT operations
    @Bean("transactionalProducerFactory")
    public ProducerFactory<String, Object> transactionalProducerFactory(Map<String, Object> kafkaProducerConfigs) {
        kafkaProducerConfigs.put(ProducerConfig.TRANSACTION_ID_PREFIX_CONFIG, "tx-producer-");
        return new DefaultKafkaProducerFactory<>(kafkaProducerConfigs);
    }
}

2. Bind Each Output to the Correct ProducerFactory #

Next, map your output1 (transactional) and output2 (non-transactional) bindings to their respective ProducerFactory beans using Spring Cloud Stream’s configuration. You can do this via YAML:

spring:
  cloud:
    stream:
      bindings:
        output1:
          destination: your-retry-dlt-topic
          producer:
            producer-factory-bean-name: transactionalProducerFactory
            # Optional: Explicitly enforce transactional behavior
            transactional: true
        output2:
          destination: your-outbox-topic
          producer:
            producer-factory-bean-name: nonTransactionalProducerFactory
            transactional: false

If you prefer Java-based configuration, use BindingsConfigurer to explicitly associate bindings with factories:

import org.springframework.cloud.stream.config.BindingsConfigurer;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class StreamBindingConfig {

    @Bean
    public BindingsConfigurer bindingsConfigurer(
            ProducerFactory<String, Object> transactionalProducerFactory,
            ProducerFactory<String, Object> nonTransactionalProducerFactory) {
        return configurer -> {
            configurer.bindProducerFactory("output1", transactionalProducerFactory);
            configurer.bindProducerFactory("output2", nonTransactionalProducerFactory);
        };
    }
}

3. Verify Behavior #

• For output2 (outbox messages): The producer will operate without transactions, aligning with your requirement to avoid wrapping the "read from JPA → send to Kafka" flow in a transaction.
• For output1 (retry/DLT transfers): The producer will use transactions, ensuring that message transfer to retry/DLT topics is atomic (either the message is moved successfully, or the operation rolls back if something fails).

Key Notes #

• Ensure your Kafka brokers are configured to support transactions (set transaction.state.log.replication.factor to at least 3, etc.).
• If you’re using plain Spring Kafka (not Spring Cloud Stream), you can achieve the same result by creating separate KafkaTemplate instances tied to each ProducerFactory, then injecting the appropriate template where needed.
• The transactional: true flag in the YAML is optional for the transactional binding, but it’s a clear way to explicitly enforce the desired behavior.

内容的提问来源于stack exchange，提问作者neko