AWS基于后端连接数的负载均衡配置及混合云流量阈值分流方案咨询

Hey there! Great question—this is a common scenario for hybrid cloud cost optimization, and I’ve helped folks set up similar flows before. Let’s break this down for you:

First, what’s this type of load balancing called? #

This pattern is most commonly referred to as Connection Threshold-Based Load Balancing, or more casually as Overflow Routing (since you’re routing excess traffic to a backup backend once your primary on-premise resource hits a connection limit). It’s a form of conditional traffic steering tied directly to resource utilization metrics.

Scalable solutions in AWS #

As you noted, AWS’s native Network Load Balancers (NLB) and Application Load Balancers (ALB) don’t support this exact threshold-based routing out of the box—they stick to weighted, least outstanding requests, or IP hash strategies. But we can build a scalable workaround using AWS services together:

1. NLB + CloudWatch Metrics + Lambda + Target Group Weight Adjustment #

This is the most serverless, cost-effective approach for most use cases:

• Set up your Target Groups: Create one Target Group (TG-OnPrem) for your on-premise servers (connected via AWS Direct Connect or Site-to-Site VPN) and another (TG-AWS) for your AWS EC2 instances. Attach both to an NLB.
• Initial routing configuration: Set TG-OnPrem’s weight to 100 and TG-AWS’s weight to 0 so all traffic goes to on-premise initially.
• Monitor connection counts: Use CloudWatch to track the ActiveConnectionCount metric for TG-OnPrem. Set up two alarms:
  • One that triggers when ActiveConnectionCount hits 950 (a buffer below your 1000 limit to avoid sudden overflow)
  • Another that triggers when it drops back to 800 (to shift traffic back to on-premise once capacity frees up)
• Automate weight adjustments: Link these alarms to a Lambda function that updates the Target Group weights via the AWS SDK. When the upper alarm fires, set TG-OnPrem to 0 and TG-AWS to 100; when the lower alarm fires, flip the weights back.

This setup scales automatically because CloudWatch and Lambda are serverless—you don’t have to manage any infrastructure to handle the monitoring or adjustment logic.

2. Deploy a Third-Party Load Balancer (like F5 BIG-IP) on AWS #

If you need more precise, real-time connection threshold control (instead of weight-based shifts), consider deploying a third-party LB like F5 BIG-IP in your AWS VPC:

• F5 supports native connection limit policies where you can define a hard cap (1000 connections) for your on-premise backend pool. Once that limit is hit, all new connections are automatically routed to your AWS backend pool.
• You can integrate F5 with AWS Direct Connect/VPN to maintain low-latency access to your on-premise resources, and it integrates seamlessly with AWS VPC networking.

3. Target Group Per-Instance Connection Limits (Limited Use Case) #

If your on-premise fleet has a fixed number of servers, you can use the ConnectionSettings feature in Target Groups to set a max connection limit per individual on-premise server. For example, if you have 2 on-premise servers, set each to 500 max connections. When both hit their limits, NLB will route new traffic to your AWS Target Group—but note this only works if you’re using the "Least Outstanding Requests" routing algorithm, and it’s per-instance rather than a global fleet limit.

Key Notes #

• The Lambda-based approach is best for most scalable, cost-focused scenarios since it leverages AWS’s managed services.
• For mission-critical workloads where you need zero-delay overflow routing, the third-party LB option is more reliable.

备注：内容来源于stack exchange，提问作者Woodgnome