Azure批量删除资源组的高效实现方案咨询

Absolutely—using multithreading (or async IO) is a fantastic way to speed up your cleanup workflow! Since deleting a resource group is an IO-bound operation (most time is spent waiting for Azure's API to process the request), your current single-threaded loop wastes valuable time waiting for each deletion to finish before moving to the next.

Core Optimization: Thread Pool for Parallel Deletions #

Python's concurrent.futures.ThreadPoolExecutor is perfect for this scenario—it manages thread lifecycles, controls concurrency limits, and simplifies error handling. Here's how to refactor your code:

Step 1: Separate Filtering from Execution #

First, extract your eligibility logic to batch-fetch all resource groups that need deletion. This decouples the "decision-making" from the "action" and makes parallel processing easier.

Step 2: Implement Parallel Deletion #

Use a thread pool to run deletion tasks in parallel. You’ll maintain your existing error-handling behavior while cutting down total runtime significantly.

Full Refactored Code Example #

from concurrent.futures import ThreadPoolExecutor, as_completed
import datetime
import traceback
from azure.core.exceptions import CloudError

# Filter resource groups to find those eligible for deletion
def get_eligible_rgs(rg_client):
    eligible_rgs = []
    required_tag = "your_required_tag_name"  # Replace with your actual mandatory tag
    
    for rg in rg_client.resource_groups.list():
        # Check for missing mandatory tag
        if required_tag not in rg.tags:
            eligible_rgs.append(rg.name)
            continue
        
        # Check if delete_at tag is expired
        delete_at_str = rg.tags.get("delete_at")
        if delete_at_str:
            try:
                delete_at = datetime.datetime.strptime(delete_at_str, "%Y-%m-%d %H:%M:%S")
                if datetime.datetime.now() > delete_at:
                    eligible_rgs.append(rg.name)
            except ValueError:
                print(f"Invalid 'delete_at' format for RG {rg.name} — skipping")
    
    return eligible_rgs

# Handle deletion of a single resource group (retains your original logic)
def delete_rg_safe(rg_client, rg_name):
    print(f"Initiating deletion of Resource Group: {rg_name}")
    try:
        delete_op = rg_client.resource_groups.delete(rg_name)
        delete_op.wait()
        print(f"Successfully deleted Resource Group: {rg_name}")
    except CloudError:
        print(f"Failed to delete Resource Group '{rg_name}'. Details:\n{traceback.format_exc()}")

# Main execution flow
def run_cleanup(rg_client):
    eligible_rgs = get_eligible_rgs(rg_client)
    
    if not eligible_rgs:
        print("No resource groups are eligible for deletion.")
        return
    
    # Control concurrency to avoid hitting Azure API rate limits
    # Start with 10-20 workers; adjust based on your subscription's quota
    max_concurrent_workers = 15
    
    with ThreadPoolExecutor(max_workers=max_concurrent_workers) as executor:
        # Submit all deletion tasks to the thread pool
        futures = {executor.submit(delete_rg_safe, rg_client, rg_name): rg_name for rg_name in eligible_rgs}
        
        # Process results as tasks complete
        for future in as_completed(futures):
            rg_name = futures[future]
            try:
                future.result()
            except Exception as e:
                print(f"Unexpected error processing {rg_name}: {str(e)}")

# Initialize your rg_client here (replace with your actual client setup)
# from azure.mgmt.resource import ResourceManagementClient
# from azure.identity import DefaultAzureCredential
# rg_client = ResourceManagementClient(DefaultAzureCredential(), "your_subscription_id")

# Run the cleanup
# run_cleanup(rg_client)

Key Considerations #

• Concurrency Limits: Azure enforces rate limits on resource management API requests. Avoid setting max_concurrent_workers too high (10-20 is a safe starting point). If you hit 429 Too Many Requests errors, add retry logic (e.g., using the tenacity library) or reduce the worker count.
• Dependency Risks: If resource groups have cross-dependencies (e.g., one RG's resources reference another), parallel deletion may cause some failures. Add retry logic for such cases or prioritize deletion order if dependencies are known.
• Async Alternative: If you’re using Azure’s async SDK, you can use asyncio for even more efficient IO handling, though this adds a bit of code complexity.
• Faster Filtering: For subscriptions with hundreds/thousands of resource groups, use Azure Resource Graph to query eligible RGs directly (instead of iterating through list()). This reduces local processing time and leverages Azure's optimized query engine.

内容的提问来源于stack exchange，提问作者David Barranco