AWS Lambda限流时S3事件积压处理咨询：队列规则与并发场景

Let’s break down your questions clearly, since this is a common pain point when dealing with high-volume S3 events and constrained Lambda concurrency:

Do S3 events get queued when Lambda is throttled? #

Yes, they do—but the queue isn’t something you can directly interact with. When S3 is configured to trigger a Lambda function, S3 maintains an internal retry queue for events that can’t be immediately processed (like when your function hits its concurrency limit). This queue is managed automatically by S3’s event notification system, not as a separate SQS queue you can inspect or configure.

How long are these queued events stored? #

By default, S3 retains unprocessed events in this internal queue for 24 hours. If an event isn’t successfully delivered and processed within that window, it will be discarded permanently. There’s no way to extend this retention period—it’s a hard limit set by AWS.

Scenario: 1M S3 POST events, Lambda concurrency set to 10 #

Here’s exactly what plays out:

• All 1 million events will be routed to S3’s internal queue immediately, since Lambda can only handle 10 concurrent executions at once.
• Lambda will pull events from this queue at its maximum allowed concurrency (10 instances running nonstop, assuming no execution failures) and process them continuously.
• The total time to clear the backlog depends entirely on your function’s per-event execution time:
  • If each event takes 1 second to process: Total time = (1,000,000 / 10) * 1s = 100,000 seconds ≈ 27.8 hours. Since this exceeds the 24-hour retention window, any events not picked up within that first day will be lost. You’d need to either increase Lambda concurrency or optimize your function to run faster to avoid data loss here.
  • If each event takes 0.5 seconds: Total time = 50,000 seconds ≈ 13.9 hours, which fits within the 24-hour window—all events would be processed successfully.

Also, remember Lambda’s maximum execution time limit: each function run can’t exceed 15 minutes (900 seconds). If your function takes longer than that per event, it will time out, and the event will be retried with exponential backoff until the 24-hour window expires. Pro tip: Make sure your function is idempotent (can safely handle duplicate events) to avoid issues from retries.

内容的提问来源于stack exchange，提问作者Nicholas Babb