实时%iowait监控脚本求助：sar结果超50触发邮件告警

Hey there! I totally get it—scripting basics can trip you up when you're just starting out, even if it seems straightforward on paper. Let's walk through exactly how to build a script that monitors %iowait with sar and shoots you an email when it crosses that 50% threshold.

Step 1: Extract %iowait from sar Output #

First, let's make sure we can pull the exact %iowait value from sar. The sar -u command gives us CPU utilization stats, including %iowait. To get a single average reading (say, over 1 second, 1 sample), run:

sar -u 1 1

You'll see output like this:

Linux 5.4.0-91-generic (your-server) 	10/05/2024 	_x86_64_	(2 CPU)

12:34:56 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
12:34:57 AM     all      0.00      0.00      0.50      2.00      0.00     97.50
Average:        all      0.00      0.00      0.50      2.00      0.00     97.50

We care about the Average line's 4th column (that's %iowait). Use awk to extract it cleanly:

sar -u 1 1 | awk '/^Average/ {print $4}'

This will spit out just the number (like 2.00 in the example).

Step 2: Build the Monitoring Script #

Create a new file (let's call it iowait_alert.sh) with the following code. I've added comments to explain each part so you can follow along easily:

#!/bin/bash

# Configure your threshold and target email address
THRESHOLD=50
ALERT_EMAIL="your-notification-email@example.com"

# Get average %iowait (adjust 1 1 to 5 3 if you want 3 samples over 5 seconds for more accuracy)
CURRENT_IOWAIT=$(sar -u 1 1 | awk '/^Average/ {print $4}')

# Convert decimal value to a whole number for simpler comparison
IOWAIT_INT=$(printf "%.0f" "$CURRENT_IOWAIT")

# Check if value exceeds threshold
if [ "$IOWAIT_INT" -gt "$THRESHOLD" ]; then
    # Compose alert content
    ALERT_SUBJECT="URGENT: High IO Wait on $(hostname)"
    ALERT_MESSAGE="IO Wait has crossed $THRESHOLD%!\nCurrent value: $CURRENT_IOWAIT%\nServer: $(hostname)\nTrigger time: $(date)"
    
    # Send the email using system mail command
    echo -e "$ALERT_MESSAGE" | mail -s "$ALERT_SUBJECT" "$ALERT_EMAIL"
fi

Step 3: Make the Script Executable #

Run this command to give the script permission to run:

chmod +x iowait_alert.sh

Step 4: Test the Script #

To verify it works, temporarily lower the threshold (like set THRESHOLD=1) and run the script:

./iowait_alert.sh

Check your email—you should get an alert if your current %iowait is over 1%. Don't forget to set the threshold back to 50 after testing!

Step 5: Automate with Cron #

To monitor non-stop, set up a cron job to run the script at regular intervals. Open your crontab editor:

crontab -e

Add this line to run the script every 5 minutes (adjust the timing to your needs):

*/5 * * * * /full/path/to/your/iowait_alert.sh

Save and exit—cron will now run the script automatically in the background.

Quick Tips to Avoid Headaches #

• Email Setup: Make sure your server has a working mail service (like Postfix or Sendmail) installed. Test it first with echo "Test alert" | mail -s "Test" your-email@example.com to ensure emails deliver correctly.
• Prevent Spam: If you don't want to get flooded with alerts, add a check to only send an email once per hour (or your preferred interval). For example, add this to your script:

  LAST_ALERT_TIMESTAMP="/tmp/last_iowait_alert.txt"
  if [ "$IOWAIT_INT" -gt "$THRESHOLD" ]; then
      # Only send alert if last one was sent over an hour ago
      if [ ! -f "$LAST_ALERT_TIMESTAMP" ] || [ $(($(date +%s) - $(cat "$LAST_ALERT_TIMESTAMP"))) -gt 3600 ]; then
          # Send email here
          date +%s > "$LAST_ALERT_TIMESTAMP"
      fi
  fi
  

内容的提问来源于stack exchange，提问作者Kumar Raj