如何将完整JSON对象存入Prometheus时序数据库并提取维度绘图

Got it, I totally get your pain—Prometheus is fantastic for numeric time-series data, but it doesn’t natively support storing full JSON objects. Luckily, there are several practical workarounds to achieve what you need: storing JSON alongside your time-series, then extracting dimensions for visualization. Let’s walk through each option:

1. Split JSON into Individual Prometheus Metrics (Most Recommended) #

This is the most straightforward approach that aligns with Prometheus’s core design. You’ll parse your JSON object in an exporter (custom or existing) and expose each numeric field as a separate Prometheus gauge/counter, with optional labels to carry non-numeric context from the JSON.

How to implement: #

• Write a simple custom exporter (e.g., using Python’s prometheus_client library) that fetches your JSON data, extracts numeric values, and exposes them as metrics.
• Add relevant labels from the JSON (like status, service_name) to keep context tied to the metrics.

Example Python Exporter: #

from prometheus_client import start_http_server, Gauge
import json
import time

# Simulate fetching your JSON data
def fetch_app_json():
    return {
        "service": "user-auth",
        "status": "healthy",
        "metrics": {
            "request_latency_ms": 45.2,
            "active_connections": 120,
            "error_rate": 0.02
        }
    }

# Define Prometheus metrics with labels
latency_gauge = Gauge('app_request_latency_ms', 'Average request latency', ['service', 'status'])
connections_gauge = Gauge('app_active_connections', 'Number of active connections', ['service', 'status'])
error_rate_gauge = Gauge('app_error_rate', 'Percentage of failed requests', ['service', 'status'])

if __name__ == '__main__':
    start_http_server(8080)  # Expose metrics on port 8080
    while True:
        json_data = fetch_app_json()
        # Update metrics with values from JSON and add labels
        latency_gauge.labels(
            service=json_data['service'],
            status=json_data['status']
        ).set(json_data['metrics']['request_latency_ms'])
        
        connections_gauge.labels(
            service=json_data['service'],
            status=json_data['status']
        ).set(json_data['metrics']['active_connections'])
        
        error_rate_gauge.labels(
            service=json_data['service'],
            status=json_data['status']
        ).set(json_data['metrics']['error_rate'])
        
        time.sleep(15)  # Scrape every 15 seconds

Pros & Cons: #

• ✅ Fully compatible with Prometheus/Grafana ecosystem—querying and visualization work out of the box
• ✅ No extra storage systems needed
• ❌ Requires maintaining the exporter if your JSON structure changes frequently
• ❌ Non-numeric JSON fields can only be stored as labels (watch out for high label cardinality, which hurts Prometheus performance)

2. Combine Prometheus with an External JSON-Compatible Store #

If you absolutely need to retain the full JSON object (not just extract numeric fields), you can send data to both Prometheus and a store that supports JSON (like Elasticsearch or InfluxDB). Then, use a visualization tool like Grafana to pull data from both sources for side-by-side comparison.

How to implement: #

• Modify your application/exporter to send numeric metrics to Prometheus, and the full JSON object to Elasticsearch (via its API) or InfluxDB (using line protocol with tags/fields).
• In Grafana, add both Prometheus and your external store as data sources. Create panels that plot Prometheus time-series alongside extracted JSON dimensions from the external store.

Pros & Cons: #

• ✅ Preserves full JSON data for future analysis
• ✅ Flexible for complex JSON structures
• ❌ Adds operational overhead (managing two storage systems)
• ❌ Visualization requires coordinating two data sources

3. Use Remote Write with a JSON-Supporting Backend #

Some time-series databases that support Prometheus’s remote write protocol (like TimescaleDB or VictoriaMetrics) allow storing additional JSON metadata alongside metrics. This lets you keep the full JSON tied to your time-series while still using Prometheus’s ingestion pipeline.

How to implement: #

• Configure Prometheus to remote write metrics to a backend like TimescaleDB, which supports jsonb columns for storing JSON metadata.
• When ingesting data, attach the full JSON as an annotation or metadata field.
• Use SQL queries (in TimescaleDB) or built-in functions (in VictoriaMetrics) to extract JSON dimensions for visualization.

Pros & Cons: #

• ✅ Keeps your existing Prometheus ingestion workflow
• ✅ Stores full JSON without extra tools
• ❌ Requires switching to a compatible backend (if you’re using vanilla Prometheus)
• ❌ Querying JSON dimensions may require learning SQL or custom query syntax

Final Recommendation #

Start with Option 1 if you only need to visualize numeric dimensions from the JSON—it’s the simplest and most efficient for Prometheus. If you must retain the full JSON object, go with Option 2 or Option 3 based on your existing infrastructure and operational preferences.

内容的提问来源于stack exchange，提问作者Brijesh Baser