如何自动删除AWS S3存储桶中名称含“run”且超过180天的文件夹？

Got it, let's tackle your two S3 cleanup problems step by step. Since S3 Lifecycle Rules don't support regex for folder name matching, we'll use practical, reliable alternative approaches tailored to your needs.

1. Automatically Delete All Files in Folders Containing "run" #

First, remember: S3 doesn't have actual folders—they're just prefixes in object keys (e.g., run01/file.txt has a prefix of run01/). The best way to handle this is a scheduled AWS Lambda function that scans and deletes matching objects.

Step-by-Step Implementation #

• Create a Lambda function (we'll use Python for readability) with this logic:
  1. List all objects in your bucket
  2. Filter objects where the "folder" prefix (first part of the key before /) contains "run"
  3. Batch delete the matching objects

Example code:

import boto3
from botocore.exceptions import ClientError

s3 = boto3.client('s3')
BUCKET_NAME = 'your-bucket-name'  # Replace with your bucket name

def lambda_handler(event, context):
    # Use a paginator to handle large buckets without timeouts
    paginator = s3.get_paginator('list_objects_v2')
    page_iterator = paginator.paginate(Bucket=BUCKET_NAME, Prefix='')
    
    objects_to_delete = []
    for page in page_iterator:
        if 'Contents' not in page:
            continue
        for obj in page['Contents']:
            # Extract the "folder" name from the object key
            folder_name = obj['Key'].split('/')[0] if '/' in obj['Key'] else obj['Key']
            if 'run' in folder_name:
                objects_to_delete.append({'Key': obj['Key']})
    
    # Delete in batches of 1000 (S3's delete limit per request)
    if objects_to_delete:
        for i in range(0, len(objects_to_delete), 1000):
            batch = objects_to_delete[i:i+1000]
            try:
                s3.delete_objects(Bucket=BUCKET_NAME, Delete={'Objects': batch})
                print(f"Deleted {len(batch)} objects from run-related folders")
            except ClientError as e:
                print(f"Error deleting objects: {e}")
    else:
        print("No objects found in run-related folders")

• Grant the Lambda function an IAM role with s3:ListBucket and s3:DeleteObject permissions for your bucket
• Schedule it to run regularly using Amazon EventBridge (e.g., daily at midnight)

2. Automatically Delete Files in "run" Folders That Are Older Than 180 Days #

This builds on the first solution—we just add a check for the object's last modified time.

Modified Lambda Code #

import boto3
from botocore.exceptions import ClientError
from datetime import datetime, timedelta, timezone

s3 = boto3.client('s3')
BUCKET_NAME = 'your-bucket-name'  # Replace with your bucket name
DAYS_THRESHOLD = 180

def lambda_handler(event, context):
    # Calculate the cutoff time (S3 uses UTC time, so we match that)
    cutoff_time = datetime.now(timezone.utc) - timedelta(days=DAYS_THRESHOLD)
    
    paginator = s3.get_paginator('list_objects_v2')
    page_iterator = paginator.paginate(Bucket=BUCKET_NAME, Prefix='')
    
    objects_to_delete = []
    for page in page_iterator:
        if 'Contents' not in page:
            continue
        for obj in page['Contents']:
            # Check if the folder contains "run"
            folder_name = obj['Key'].split('/')[0] if '/' in obj['Key'] else obj['Key']
            if 'run' not in folder_name:
                continue
            # Check if the object is older than 180 days
            if obj['LastModified'] < cutoff_time:
                objects_to_delete.append({'Key': obj['Key']})
    
    # Batch delete matching objects
    if objects_to_delete:
        for i in range(0, len(objects_to_delete), 1000):
            batch = objects_to_delete[i:i+1000]
            try:
                s3.delete_objects(Bucket=BUCKET_NAME, Delete={'Objects': batch})
                print(f"Deleted {len(batch)} old objects from run-related folders")
            except ClientError as e:
                print(f"Error deleting objects: {e}")
    else:
        print("No old objects found in run-related folders")

How This Works for Your Example #

For your buckets with run01/, run02/, run03/, and name01/:

• The code skips all objects in name01/ entirely
• It only targets objects in the three run* folders
• Objects in those folders are only deleted if their last modified date is more than 180 days in the past

Quick Tips #

• Test the Lambda function with a small bucket or subset of objects first to verify logic
• For extremely large buckets, add error handling and consider increasing Lambda's timeout limit
• If you need real-time tracking (e.g., flagging new objects for deletion in 180 days), you could pair S3 Event Notifications with a database to track upload times, but scheduled batch scans are simpler for most use cases

内容的提问来源于stack exchange，提问作者Nitin Punekar