咨询AWS Lambda最大出站连接数及文件描述符限制范围

Let’s break this down clearly—this is a super common pain point when dealing with high-concurrency Lambda workloads, so your hunch about Lambda’s limits is totally valid to investigate:

1. Lambda’s File Descriptor Limit: Per Instance, Not Total #

The 1024 file descriptor limit you found in AWS docs applies to each individual Lambda invocation instance, not across all running Lambda instances in your account or region.

Every Lambda execution environment (the isolated runtime that runs your function for a single invocation) gets its own isolated set of resources, including this 1024 file descriptor quota. A small chunk of these descriptors are already used by the Lambda runtime itself (for internal logging, runtime API sockets, etc.), so the actual number of available outbound TCP connections you can initiate from one instance is slightly less—usually around 900-950, depending on your chosen runtime.

2. Why You’re Hitting a ~1024 Concurrent Connection Bottleneck #

If your Kinesis-processing Lambda is hitting this ~1024 connection limit, it’s almost certainly tied to the per-instance file descriptor cap:

• If your function is processing large Kinesis batches (e.g., 1000+ records per invocation) and spawning concurrent HTTP requests for each record (like using Promise.all with a huge array of calls), you’ll hit the file descriptor wall fast—each TCP connection consumes one descriptor.
• Unlike EC2 instances, this limit is hard and non-configurable for Lambda—you can’t bump it up.

3. Troubleshooting & Fixes to Try #

Here’s how to confirm the issue and work around it:

• Check if you’re overloading single Lambda instances: Look at your Lambda invocation logs to see if you’re processing massive Kinesis batches. If yes, reduce the batch size in your Kinesis event source mapping—this splits work across more Lambda instances, each handling fewer concurrent requests.
• Log open file descriptors to confirm: In your Lambda function (for Linux-based runtimes like Node.js, Python, Java), add code to log the number of open file descriptors. For example, in Python run len(os.listdir('/proc/self/fd')) and log the value. If this number climbs close to 1024 when you hit the bottleneck, you’ve confirmed the root cause.
• Rule out other bottlenecks: Double-check your web endpoint’s load balancer or backend limits. An Application Load Balancer (ALB) handles tens of thousands of concurrent connections by default, so it’s unlikely to be the issue unless you’ve customized its settings—but it’s worth verifying to eliminate variables.
• Use connection pooling: If you’re making repeated requests to the same web endpoint, implement connection pooling (e.g., axios with a pool in Node.js, urllib3 pooling in Python). This reuses existing TCP connections instead of creating new ones, cutting down on file descriptor usage drastically.

内容的提问来源于stack exchange，提问作者justin.m.chase