AWS Lambda发送请求后超时及重复POST请求问题求助

Hey there, let's work through your problem step by step. You're hitting two big pain points: your Lambda is timing out after 30 seconds, and its retry mechanism is spamming duplicate POST requests. On top of that, your current code has some gaps in error handling that are making things worse. Let's tackle each issue one by one.

1. Fix the Lambda Timeout Issue #

First off, that 30-second timeout is likely happening because you're trying to process too many requests at once with Promise.all, which can overwhelm both Lambda and your target API. Here's how to fix it:

• Bump up Lambda's timeout limit: AWS Lambda lets you set timeouts up to 15 minutes. Head to your Lambda function's configuration and increase the timeout to a value that makes sense for your workload (e.g., if each POST takes 1 second and you have 100 shipments, set it to 2 minutes to be safe).
• Process shipments in batches: Instead of firing all POST requests at once, split your shipment array into smaller batches (like 10 items per batch). This reduces concurrent load on your target API and prevents Lambda from hitting its execution limit too quickly.
• Optimize request efficiency: Use an axios instance with connection pooling (keep-alive) to reuse HTTP connections, and set a reasonable timeout for individual requests so a single stuck request doesn't take down the whole function.

2. Stop Duplicate Requests From Retries #

Lambda's retry behavior (up to 5 times for async invocations) means every timeout triggers a full re-run of your function, re-sending all POSTs. To fix this:

• Make your requests idempotent: Work with the team that owns your target API to ensure POST requests can be safely retried. Include a unique identifier (like item.order_reference combined with data.batchID) in each request. The API should ignore duplicate requests with the same identifier instead of creating duplicate resources.
• Track processed shipments: Use a DynamoDB table to log which shipments have been successfully processed. Before sending a POST, check if the shipment's order reference already exists in the table. If it does, skip it. This way, retries only handle unprocessed items.
• Tweak Lambda retry settings: If you're using async invocation, you can reduce the number of retries in Lambda's configuration or route failed invocations to a dead-letter queue (DLQ) for manual review instead of automatic retries.

3. Fix Your Code's Error Handling #

Your current code has commented-out logic for collecting successes/failures, and Promise.all will throw an error immediately if any single POST fails—triggering a retry even if most requests succeeded. Here's an improved version of your code:

module.exports.shipments = async (event) => {
  const axios = require("axios");
  const http = require('http');
  const https = require('https');

  // Create an axios instance with connection pooling and per-request timeout
  const axiosInstance = axios.create({
    timeout: 10000, // Time out individual requests after 10 seconds
    httpAgent: new http.Agent({ keepAlive: true }),
    httpsAgent: new https.Agent({ keepAlive: true })
  });

  let data = JSON.parse(event.body);
  let url = `${data.apiURL}/api/1.1/wf/bulkshipments`;

  let good = [];
  let bad = [];

  // Process shipments in batches of 10 (adjust based on your API's rate limits)
  const batchSize = 10;
  for (let i = 0; i < data.shipments.length; i += batchSize) {
    const batch = data.shipments.slice(i, i + batchSize);
    const batchPromises = batch.map(async (item) => {
      try {
        const res = await axiosInstance.post(url, {
          batchID: data.batchID,
          companyID: data.companyID,
          shipment: item
        });
        good.push({ "Order Reference": item.order_reference, "result": res.data });
      } catch (error) {
        // Capture error details (fallback to error message if no response data)
        const errorDetails = error.response?.data || error.message;
        bad.push({ "Order Reference": item.order_reference, "result": errorDetails });
      }
    });

    // Wait for the current batch to finish before moving to the next
    await Promise.all(batchPromises);
  }

  // Return 207 if some requests failed, 200 if all succeeded
  return {
    statusCode: bad.length > 0 ? 207 : 200,
    body: JSON.stringify({
      message: bad.length > 0 ? 'Partial success' : 'Success',
      totalShipments: data.shipments.length,
      created: good.length,
      errors: bad.length,
      result: { completed: good, failed: bad }
    }, null, 2),
  };
};

Key improvements in this code: #

• Uses an axios instance with persistent connections to speed up requests
• Processes shipments in batches to avoid overwhelming Lambda/your API
• Wraps each POST in a try/catch so individual failures don't crash the whole function
• Returns a 207 status code for partial successes, making it clearer to the caller what went wrong
• Properly collects both successful and failed request results

Extra Tips #

• Monitor with CloudWatch: Track Lambda's execution time, error rates, and concurrency to spot bottlenecks early.
• Add exponential backoff for retries: If your target API rate-limits you, add retry logic with exponential backoff inside the catch block (instead of relying on Lambda's global retries).
• Clean up unused code: The patchURL variable in your original code isn't used—remove it to keep things tidy.

内容的提问来源于stack exchange，提问作者Shawn C.