RabbitMQ 3.8.0 Prometheus插件获取指标返回204 No Content及badarg错误的排查求助

Let’s work through this issue step by step—you’re hitting a 204 No Content response with underlying 500 errors and badarg process exits when trying to access the Prometheus metrics endpoint on your RabbitMQ 3.8.0 instance (running on Erlang 23.3.4.11). Here are the most actionable fixes to resolve this:

1. Reset Prometheus Plugins to Fix Cached State Conflicts #

Old or misloaded plugin state can cause compatibility issues with newer Erlang versions. Let’s fully reset the plugin setup:

• First, disable the prometheus-related plugins and shut down RabbitMQ:

  rabbitmq-plugins disable rabbitmq_prometheus prometheus
  rabbitmq-server stop
  

• Wait 10-15 seconds for the node to fully shut down, then restart RabbitMQ:

  rabbitmq-server start
  

• Re-enable the prometheus plugin set cleanly:

  rabbitmq-plugins enable rabbitmq_prometheus
  

This ensures plugins load fresh without leftover cached data causing unexpected crashes.

2. Enable Debug Logging to Capture Full Error Context #

Your current logs only show badarg with an empty stacktrace—this isn’t enough to pinpoint exactly what’s failing. Let’s enable debug-level logging to get detailed error traces:

• Edit your rabbitmq.conf (located at /etc/rabbitmq/rabbitmq.conf) and add:

  log.level = debug
  

• Restart RabbitMQ, then send another request to the metrics endpoint:

  curl http://localhost:15692/metrics
  

• Check the logs again—you’ll now see a full stacktrace that shows which function is throwing the badarg error. This will help you confirm if it’s a plugin bug, configuration mistake, or Erlang compatibility issue.

3. Align Erlang/RabbitMQ Versions for Better Compatibility #

RabbitMQ 3.8.0 supports Erlang 21.3.x to 23.x, but some minor Erlang releases (like 23.3.4.11) can have edge-case incompatibilities with older RabbitMQ versions. Consider two options:

• Downgrade Erlang: Switch to a stable release explicitly recommended for RabbitMQ 3.8.0, such as Erlang 23.2.x
• Upgrade RabbitMQ: Move to a newer 3.8.x patch release (like 3.8.28, the latest in the 3.8 series) — this version includes fixes for Prometheus plugin bugs and improved Erlang 23 support.

4. Validate Prometheus Endpoint Configuration #

Double-check that your RabbitMQ config doesn’t have invalid settings breaking the metrics endpoint:

• Ensure there’s no misconfigured prometheus.path in rabbitmq.conf — the default /metrics should work, so remove any custom path if you added one.
• Confirm the 15692 listener is properly bound (your diagnostic output shows it’s listening on [::]:15692, which is correct for IPv6/IPv4 compatibility).

5. Rule Out Sudden Resource Spikes #

While your diagnostic data shows healthy memory and disk usage, occasional resource spikes can cause unexpected process crashes. Monitor RabbitMQ’s real-time metrics via the management UI at http://localhost:15672 while hitting the Prometheus endpoint to check for sudden jumps in memory or CPU usage.

内容的提问来源于stack exchange，提问作者Metin Bulak