如何在Prometheus中运行单行Shell脚本并获取指定挂载点下一级目录的磁盘占用数据？

Great question! You're absolutely right that the standard Node Exporter filesystem metrics like node_filesystem_avail_bytes only expose data at the mount point level. But there's a straightforward way to get the per-subdirectory disk usage (like your du command outputs) into Prometheus, and then visualize it in Grafana. Here's how to do it:

1. Create a Script to Generate Prometheus-Format Metrics #

First, we'll write a bash script that runs your du logic and converts the output into Prometheus-compatible metrics. We'll use raw byte counts instead of human-readable formats since Prometheus works best with numeric values.

Save this as subdir_disk_usage.sh (replace /your/target/mount with your actual mount point):

#!/bin/bash
# Define your target mount point
MOUNT_POINT="/your/target/mount"

# Run du to get raw byte counts for first-level subdirectories
du -b --max-depth=1 "$MOUNT_POINT" | \
  # Filter out the mount point itself (we only want subdirs)
  grep -v "^$(du -b "$MOUNT_POINT" | awk '{print $1}')\s$MOUNT_POINT$" | \
  while read -r SIZE DIR; do
    # Convert directory names to valid Prometheus label characters (replace specials with underscores)
    CLEAN_DIR=$(echo "$DIR" | sed 's/[^a-zA-Z0-9\/]/_/g')
    # Output the metric with mountpoint and directory labels
    echo "node_filesystem_subdir_size_bytes{mountpoint=\"$MOUNT_POINT\",directory=\"$CLEAN_DIR\"} $SIZE"
done

Make the script executable:

chmod +x /path/to/subdir_disk_usage.sh

2. Use Node Exporter's Textfile Collector #

Node Exporter has a built-in textfile collector that lets you import custom metrics from static files. Here's how to set it up:

• When starting Node Exporter, add the --collector.textfile.directory flag to specify where your metric files will live:

  node_exporter --collector.textfile.directory=/var/lib/node_exporter/textfile_collector
  

  (Create the directory first if it doesn't exist: mkdir -p /var/lib/node_exporter/textfile_collector)

• Set up a cron job to run your script regularly and update the metric file. This ensures your data stays fresh. Run crontab -e and add:

  # Update subdirectory disk usage every 5 minutes
  */5 * * * * /path/to/subdir_disk_usage.sh > /var/lib/node_exporter/textfile_collector/subdir_disk_usage.prom.$$ && mv /var/lib/node_exporter/textfile_collector/subdir_disk_usage.prom.$$ /var/lib/node_exporter/textfile_collector/subdir_disk_usage.prom
  

  The .$$ suffix and mv command ensure we don't write an incomplete file to the collector directory (this is an atomic update to avoid scraping partial data).

3. Query the Metrics in Prometheus #

Once the script is running and Node Exporter is scraping the textfile directory, you can query the new metric in Prometheus:

• To get raw byte counts for subdirectories under your mount point:

  node_filesystem_subdir_size_bytes{mountpoint="/your/target/mount"}
  

• To convert to human-readable format (e.g., GB, MB) for easier reading:

  bytes(node_filesystem_subdir_size_bytes{mountpoint="/your/target/mount"})
  

4. Visualize in Grafana #

In Grafana, you can add a panel to display this data:

• For a bar chart: Use the query above, set the "Legend" to {{directory}} so each bar represents a subdirectory, and set the unit to "bytes" (Grafana will auto-convert to appropriate units like GB).
• For a table: Use the same query, add columns for directory and the metric value, and format the value column to show human-readable bytes.

Key Notes #

• Permissions: Ensure the user running the cron job has read access to all subdirectories under your mount point, and write access to the textfile collector directory.
• Multiple Mount Points: If you need to monitor multiple mount points, modify the script to loop through a list of mount points, or create separate scripts for each.
• Frequency: Adjust the cron interval based on how often you need fresh data (e.g., * * * * * for every minute, or */10 * * * * for every 10 minutes).

内容的提问来源于stack exchange，提问作者vinWin