Spring Boot 2中如何将Kafka指标暴露至/actuator/metrics

Great question! Let's break this down for your Spring Boot 2 + Spring Kafka 2.1.4 setup:

Are Kafka consumer metrics out-of-the-box? #

Short answer: Not fully out-of-the-box, but they're easy to enable with minimal configuration.

Spring Kafka 2.1.x introduced integration with Micrometer (the metrics library powering Spring Boot 2's Actuator), so core consumer metrics are available once you flip the right switches.

Step 1: Enable Default Metrics #

First, make sure you have the required dependencies in your pom.xml (or build.gradle):

• spring-boot-starter-actuator (for the /metrics endpoint)
• spring-kafka (version 2.1.4, which you're already using)
• Micrometer core is included via the Actuator starter, so no need to add it separately.

Next, configure your application properties to expose the metrics endpoint and enable Kafka metrics collection:

# Expose the metrics endpoint via HTTP
management.endpoints.web.exposure.include=metrics
# Enable Micrometer's Kafka consumer metrics
management.metrics.enable.kafka=true

Once this is set up, Spring Kafka will automatically bind metrics to the Micrometer registry for your @KafkaListener containers. You can access these via the Actuator endpoint:

• Visit /metrics to see all available metric keys (look for ones prefixed with kafka.consumer.*)
• To see the actual value of a specific metric, use /metrics/kafka.consumer.records.consumed.total

Some of the default metrics you'll get:

• kafka.consumer.records.consumed.total: Total number of records consumed
• kafka.consumer.fetch.records.total: Total number of records fetched
• kafka.consumer.records.lag: Current lag per partition
• kafka.consumer.commit.latency.avg: Average latency of offset commits

Step 2: Custom Metrics (If You Need More) #

If the default metrics aren't enough (e.g., you want topic-specific counts, custom error metrics, or latency tracking), the best approach is to use a ConsumerInterceptor or leverage Micrometer's MeterRegistry directly.

Example: Custom Consumer Interceptor for Topic-Specific Metrics #

Create an interceptor to track consumed records per topic/partition:

@Component
public class TopicPartitionMetricsInterceptor implements ConsumerInterceptor<String, Object> {

    private final MeterRegistry meterRegistry;

    // Inject MeterRegistry automatically (provided by Spring Boot)
    public TopicPartitionMetricsInterceptor(MeterRegistry meterRegistry) {
        this.meterRegistry = meterRegistry;
    }

    @Override
    public ConsumerRecords<String, Object> onConsume(ConsumerRecords<String, Object> records) {
        // Count records per topic and partition
        records.partitions().forEach(partition -> {
            long recordCount = records.records(partition).size();
            meterRegistry.counter("kafka.consumer.topic.partition.records.consumed",
                    "topic", partition.topic(),
                    "partition", String.valueOf(partition.partition()))
                    .increment(recordCount);
        });
        return records;
    }

    // Default implementations for other interceptor methods
    @Override
    public void onCommit(Map<TopicPartition, OffsetAndMetadata> offsets) {}

    @Override
    public void close() {}

    @Override
    public void configure(Map<String, ?> configs) {}
}

Then attach this interceptor to your Kafka listener container factory:

@Bean
public ConcurrentKafkaListenerContainerFactory<String, Object> kafkaListenerContainerFactory(
        ConsumerFactory<String, Object> consumerFactory,
        TopicPartitionMetricsInterceptor metricsInterceptor) {
    ConcurrentKafkaListenerContainerFactory<String, Object> factory =
            new ConcurrentKafkaListenerContainerFactory<>();
    factory.setConsumerFactory(consumerFactory);
    
    // Add the custom interceptor to the container
    factory.getContainerProperties().setInterceptors(List.of(metricsInterceptor));
    return factory;
}

Now you'll have a custom metric kafka.consumer.topic.partition.records.consumed that you can query via the Actuator endpoint, filtered by topic and partition.

Key Notes #

• Remember that in Spring Boot 2, the /metrics endpoint only lists metric keys by default—you need to append the metric name (like /metrics/kafka.consumer.records.consumed.total) to see the actual value.
• If you're using YAML instead of properties, adjust the configuration syntax accordingly (e.g., management.endpoints.web.exposure.include: metrics).

内容的提问来源于stack exchange，提问作者Michael Azimov