API异步请求适配请求速率限制方案咨询

Hey there! Let's break down how to solve this rate limiting and retry issue properly. Since your project uses Node.js primarily, I'll start with that, then cover Python as a backup option.

Node.js Implementation #

The key here is to use a rate limiter that enforces 10 requests per second (not just 10 concurrent) and add retry logic for "Limit Exceed" errors. We'll use two popular libraries: bottleneck (for rate/concurrency control) and p-retry (for retries with exponential backoff).

Step 1: Install Dependencies #

npm install bottleneck p-retry node-fetch

Step 2: Code Example #

const Bottleneck = require('bottleneck');
const pRetry = require('p-retry');
const fetch = require('node-fetch');

// Initialize rate limiter: 10 requests per second (100ms between each request)
const limiter = new Bottleneck({
  minTime: 100, // 100ms = 10 requests/sec
  maxConcurrent: 10, // Optional: cap concurrent requests to avoid overwhelming the API
});

// Define your API fetch function with error handling
const fetchTicket = async (ticketId) => {
  const response = await fetch(`https://your-api-url.com/tickets/${ticketId}`);
  
  // Check for rate limit error (adjust status code/message based on your API's response)
  if (response.status === 429 || (await response.text()).includes('Limit Exceed')) {
    throw new pRetry.AbortError('Rate limit exceeded'); // Trigger retry
  }
  
  if (!response.ok) {
    throw new Error(`Request failed with status ${response.status}`);
  }
  
  return response.json();
};

// Wrap fetch function with retry logic
const fetchWithRetry = async (ticketId) => {
  return pRetry(() => fetchTicket(ticketId), {
    retries: 3, // Number of retries before giving up
    factor: 2, // Exponential backoff: 1s, 2s, 4s between retries
    minTimeout: 1000, // Minimum wait time before first retry
    onFailedAttempt: (error) => {
      console.log(`Attempt ${error.attemptNumber} failed for ticket ${ticketId}: ${error.message}`);
    },
  });
};

// Process all tickets with rate limiting and retries
const processAllTickets = async (ticketIds) => {
  // Wrap the retry-enabled function with the rate limiter
  const limitedFetch = limiter.wrap(fetchWithRetry);
  
  // Run all requests concurrently (but rate-limited)
  const results = await Promise.all(ticketIds.map(limitedFetch));
  
  // Filter out any failed requests (if needed)
  const successfulResults = results.filter(result => result !== undefined);
  
  return successfulResults;
};

// Usage example:
// const ticketIds = [1, 2, 3, ..., 500];
// processAllTickets(ticketIds).then(results => console.log('Done:', results));

Python Implementation #

For Python, we'll use aiohttp for async requests, tenacity for retries, and a custom semaphore + timestamp tracker to enforce the 10 requests/sec limit.

Step 1: Install Dependencies #

pip install aiohttp tenacity

Step 2: Code Example #

import asyncio
import aiohttp
from tenacity import retry, stop_after_attempt, wait_exponential, retry_if_exception_type

class RateLimitError(Exception):
    """Custom exception for rate limit hits"""
    pass

async def fetch_ticket(session: aiohttp.ClientSession, ticket_id: int):
    url = f"https://your-api-url.com/tickets/{ticket_id}"
    async with session.get(url) as response:
        response_text = await response.text()
        
        # Check for rate limit error (adjust based on your API's response)
        if response.status == 429 or "Limit Exceed" in response_text:
            raise RateLimitError("API rate limit exceeded")
        
        response.raise_for_status()
        return await response.json()

# Add retry logic with exponential backoff
@retry(
    stop=stop_after_attempt(3),
    wait=wait_exponential(multiplier=1, min=1, max=10),
    retry=retry_if_exception_type(RateLimitError),
    before_sleep=lambda state: print(f"Retrying ticket {state.args[0]} (attempt {state.attempt_number})...")
)
async def fetch_with_retry(session: aiohttp.ClientSession, ticket_id: int):
    return await fetch_ticket(session, ticket_id)

async def process_all_tickets(ticket_ids: list[int]):
    semaphore = asyncio.Semaphore(10)
    request_timestamps = []
    
    async def limited_fetch(ticket_id: int):
        async with semaphore:
            now = asyncio.get_event_loop().time()
            
            # Remove timestamps older than 1 second
            request_timestamps[:] = [t for t in request_timestamps if now - t < 1]
            
            # If we've hit 10 requests in the last second, wait until we can send more
            if len(request_timestamps) >= 10:
                wait_time = 1 - (now - request_timestamps[0])
                await asyncio.sleep(wait_time)
                now = asyncio.get_event_loop().time()
                request_timestamps[:] = [t for t in request_timestamps if now - t < 1]
            
            request_timestamps.append(now)
            return await fetch_with_retry(session, ticket_id)
    
    async with aiohttp.ClientSession() as session:
        tasks = [limited_fetch(tid) for tid in ticket_ids]
        results = await asyncio.gather(*tasks, return_exceptions=True)
    
    # Filter out exceptions to get only successful results
    successful_results = [res for res in results if not isinstance(res, Exception)]
    return successful_results

# Usage example:
# ticket_ids = list(range(1, 501))
# asyncio.run(process_all_tickets(ticket_ids))

Best Practices to Optimize Further #

• Check for Batch API Endpoints: Many APIs let you fetch multiple resources in one request (e.g., /tickets?ids=1,2,3). If this is available, it'll cut your request count from 500 to just a handful, which is way more efficient.
• Respect Rate Limit Headers: Most APIs return headers like X-RateLimit-Remaining and X-RateLimit-Reset. Use these to dynamically adjust your request rate instead of hardcoding 10/sec—this avoids hitting limits unnecessarily.
• Log Failed Requests: Keep track of tickets that failed even after retries so you can manually reprocess them later.
• Adjust Retry Parameters: Tweak the number of retries and backoff time based on how the API behaves. Some APIs might have longer cooldown periods after hitting limits.

内容的提问来源于stack exchange，提问作者Matheus Leonardo