Elastic APM Python代理进程指标未在Kibana显示的技术求助

Let’s walk through your issue step by step, based on your Python 3.7 + elastic-apm 5.8.0 + Elastic Stack 7.7.1 setup:

First: Are Process Memory, Uptime, and Thread Count Metrics Being Collected? #

The good news is that the elastic-apm Python agent 5.8.0 does collect these metrics by default—they just aren’t included in the out-of-the-box APM dashboard in Kibana. Here’s how to confirm and visualize them:

1. Verify the Data is Reaching Elasticsearch #

Head to Kibana’s Discover tab, select the apm-* index pattern, and run these searches:

• For process metrics: service.name:MY_SERVICE AND metricset.name:process
  Look for fields like process.memory.rss (resident memory), process.uptime (process run time), and process.threads.count (thread count).
• For GC metrics: service.name:MY_SERVICE AND metricset.name:gc
  You should see gc.collections (count of garbage collections per generation) if you’re using CPython.

If these fields show up, the data is being collected correctly—you just need to add custom visualizations to see them.

2. Fix Potential Configuration Issues #

First, correct a couple of small mistakes in your client initialization code (these might be blocking some metrics):

client = elasticapm.Client(
    service_name="MY_SERVICE",
    service_node_name="MY_SERVICE_NODE",
    server_url="http://apm-server:8200",  # Removed the extra "=" in your original URL
    recording=True,  # Use a boolean value instead of string "true"
    environment="PROD",
    metrics_interval=10  # Default is 10s; ensures metrics are sent regularly
)
elasticapm.instrument()

Ensure you don’t have a disable_metrics parameter excluding process.* or gc.* metrics—this would block collection entirely.

3. Create Custom Visualizations in Kibana #

Since the default APM dashboard doesn’t include these metrics, build your own:

1. Go to Visualize Library → Create visualization → Pick a type (e.g., Line Chart for trends, Metric for single values).
2. Select the apm-* index pattern.
3. Configure the visualization:
   • For process memory: Set Y-axis to process.memory.rss (aggregation: Average), X-axis to @timestamp (Date Histogram). Add filters for service.name:MY_SERVICE and environment:PROD.
   • For thread count: Use process.threads.count instead.
4. Save the visualization, then add it to your existing APM service dashboard (Edit Dashboard → Add visualization) or create a new custom dashboard.

Second: Advanced Metrics (Thread-Level CPU, IO Stats) #

Unfortunately, these metrics are not supported by the elastic-apm Python agent 5.8.0 (the version aligned with your Elastic Stack 7.7.1):

• Thread-level CPU usage: Python’s process model and lack of built-in thread-level CPU tracking make this unfeasible for the agent to collect, unlike Java’s JVM which exposes this natively.
• IO statistics: The Python agent does not collect process-level disk/network IO metrics in this version—this is a feature exclusive to the Java agent (and some other language agents) at the time of 7.7.1.
• GC statistics: As noted earlier, basic GC collection counts are collected, but more granular GC metrics (like pause time) are not available in this agent version.

Summary #

• Available metrics (process memory, uptime, thread count, GC collection counts): Confirm they exist in Discover, fix any configuration typos, and build custom visualizations in Kibana to display them.
• Unsupported metrics (thread-level CPU, IO stats): These aren’t available in your agent version; you’d need to use external tools (like psutil in your code) to collect and send them as custom metrics to Elastic APM if needed.

内容的提问来源于stack exchange，提问作者Debashish BHARALI