如何扫描AWS账户中生命周期过期超X天的S3存储桶并发送告警邮件？

Let's break this down into two practical parts: first verifying if a single bucket has any lifecycle rules with expiration days exceeding your threshold, then scaling that to scan all buckets in your AWS account and trigger email alerts when violations are found.

1. 单个存储桶的过期规则检查 #

Your original command lists all expiration days, but we can enhance it with jq to directly confirm if any rule crosses your threshold X. Here's the refined command:

# Replace X with your threshold (e.g., 365) and Bucket_Name with your target bucket
aws s3api get-bucket-lifecycle --bucket Bucket_Name 2>/dev/null | jq 'any(.Rules[]?.Expiration?.Days; . > X)'

关键细节： #

• 2>/dev/null suppresses errors if the bucket has no lifecycle configuration (prevents the command from failing abruptly).
• The ? in .Rules[]?.Expiration?.Days safely handles rules that don’t include an Expiration setting (avoids null reference errors).
• jq's any() function returns true if at least one rule meets the condition (Days > X), otherwise false.

If you want to see the exact violating rules (instead of just a boolean), use this variant:

aws s3api get-bucket-lifecycle --bucket Bucket_Name 2>/dev/null | jq '.Rules[] | select(.Expiration?.Days > X)'

2. 批量扫描所有S3存储桶并发送告警 #

We can wrap the single-bucket check into a shell script that iterates over all your buckets, logs violations, and sends an alert email.

步骤1：编写扫描脚本 #

Create a file named s3-expiration-scan.sh with this content:

#!/bin/bash

# Configuration
THRESHOLD_DAYS=365  # Replace with your desired threshold
ALERT_EMAIL="your-alert-email@example.com"
TEMP_LOG="/tmp/s3-expiration-violations.log"

# Clear previous log
> "$TEMP_LOG"

# Get list of all S3 buckets
BUCKETS=$(aws s3api list-buckets --query 'Buckets[].Name' --output text)

for BUCKET in $BUCKETS; do
    echo "Checking bucket: $BUCKET"
    
    # Check if any rule exceeds threshold
    HAS_VIOLATION=$(aws s3api get-bucket-lifecycle --bucket "$BUCKET" 2>/dev/null | jq -r "any(.Rules[]?.Expiration?.Days; . > $THRESHOLD_DAYS)")
    
    # Handle buckets with no lifecycle config (HAS_VIOLATION will be "null")
    if [ "$HAS_VIOLATION" = "true" ]; then
        # Fetch violating rules for detailed logging
        VIOLATING_RULES=$(aws s3api get-bucket-lifecycle --bucket "$BUCKET" | jq -r '.Rules[] | select(.Expiration?.Days > '"$THRESHOLD_DAYS"') | "Rule Name: \(.ID), Expiration Days: \(.Expiration.Days)"')
        
        echo -e "\n=== Violation found in bucket: $BUCKET ===" >> "$TEMP_LOG"
        echo "$VIOLATING_RULES" >> "$TEMP_LOG"
    fi
done

# Send email if violations exist
if [ -s "$TEMP_LOG" ]; then
    SUBJECT="S3 Expiration Alert: Buckets with expiration days exceeding $THRESHOLD_DAYS"
    BODY="The following S3 buckets have lifecycle rules with expiration days beyond your threshold:\n\n$(cat "$TEMP_LOG")"
    
    # Option 1: Use Linux mailx (requires mailutils package)
    echo -e "$BODY" | mailx -s "$SUBJECT" "$ALERT_EMAIL"
    
    # Option 2: Use AWS SES (more reliable for cloud environments)
    # aws ses send-email --from "sender-email@example.com" --destination "ToAddresses=$ALERT_EMAIL" --message "Subject={Data=$SUBJECT,Charset=utf8},Body={Text={Data=$BODY,Charset=utf8}}"
fi

# Clean up temp log
rm "$TEMP_LOG"

步骤2：配置权限 #

Ensure the IAM entity running this script has these permissions:

• s3:ListAllMyBuckets (to retrieve the full bucket list)
• s3:GetBucketLifecycleConfiguration (to read lifecycle rules for each bucket)
• If using AWS SES: ses:SendEmail (to send alert emails)

步骤3：运行脚本 #

Make the script executable and run it:

chmod +x s3-expiration-scan.sh
./s3-expiration-scan.sh

可选：定时自动扫描 #

To run this check daily (or on your preferred schedule), add a cron job:

# Open crontab editor
crontab -e

# Add this line to run at 9 AM every day (adjust time as needed)
0 9 * * * /path/to/s3-expiration-scan.sh

额外注意事项 #

• Some buckets might use Expiration.Date instead of Days for lifecycle rules. To include these in your check, modify the jq query to handle both fields:

  jq 'any(.Rules[]?.Expiration; (.Days > X) or (.Date | fromdate > (now + X*86400)))'
  

• If you have hundreds of buckets, add a small delay (e.g., sleep 1) inside the loop to avoid hitting AWS API rate limits.

内容的提问来源于stack exchange，提问作者Tarun Prakash