粘性会话与WebSocket场景下的弹性扩缩容问题及方案咨询

Great question—this is a super common pain point with WebSocket-based apps behind load balancers, especially when dealing with spiky traffic like workshop launches. Let’s break down your options step by step, focusing on practical, production-ready solutions.

This is the most scalable approach for your scenario, as it decouples client connections from specific EC2 instances. Here’s how it works:

• Centralize connection state in Redis: Every Node.js instance connects to a shared Redis cluster (use Amazon ElastiCache for managed Redis on AWS). When a client establishes a WebSocket connection to any instance, the instance stores the client’s identity (e.g., user ID) and connection metadata (instance ID, local connection ID) in a Redis Hash.
• Route messages via Redis Pub/Sub: When you need to send a message to a specific user, first look up their active instance from Redis. Then publish the message to a Redis channel dedicated to that instance. The target instance subscribes to its channel, receives the message, and pushes it to the connected client.
• Load balancing without stickiness: Your ALB can now distribute all traffic (HTTP and WebSocket upgrade requests) evenly across all EC2 instances—no more new instances sitting idle during workshop spikes.

Quick Code Example (Node.js + ioredis) #

const Redis = require('ioredis');
const WebSocket = require('ws');
const { v4: uuidv4 } = require('uuid');

// Initialize Redis clients (one for commands, one for subscription per connection)
const redis = new Redis({ host: process.env.REDIS_HOST });
const wss = new WebSocket.Server({ port: 8080 });

// Local map to track active connections on this instance
const localConnections = new Map();

wss.on('connection', async (ws, req) => {
  // Extract user ID from request (e.g., auth token query param)
  const userId = req.url.split('?userId=')[1];
  const connectionId = uuidv4();
  const instanceId = process.env.EC2_INSTANCE_ID; // Get from AWS metadata

  // Store connection mapping in Redis
  await redis.hset(`user:${userId}`, {
    instanceId,
    connectionId
  });

  // Track connection locally
  localConnections.set(connectionId, ws);

  // Subscribe to this instance's dedicated Redis channel
  const subscriber = new Redis({ host: process.env.REDIS_HOST });
  await subscriber.subscribe(`instance:${instanceId}`);

  subscriber.on('message', (_, msg) => {
    const { targetConnectionId, payload } = JSON.parse(msg);
    const targetWs = localConnections.get(targetConnectionId);
    if (targetWs && targetWs.readyState === WebSocket.OPEN) {
      targetWs.send(payload);
    }
  });

  // Cleanup on connection close
  ws.on('close', async () => {
    localConnections.delete(connectionId);
    await redis.del(`user:${userId}`);
    await subscriber.unsubscribe();
    subscriber.quit();
  });
});

// Helper function to send messages to a user
async function sendToUser(userId, payload) {
  const userConnection = await redis.hgetall(`user:${userId}`);
  if (!userConnection.instanceId || !userConnection.connectionId) return;

  await redis.publish(`instance:${userConnection.instanceId}`, JSON.stringify({
    targetConnectionId: userConnection.connectionId,
    payload: JSON.stringify(payload)
  }));
}

If you prefer to keep sticky sessions but want to offload traffic from overloaded instances, you can implement session migration:

• Trigger migration on high load: Use CloudWatch metrics to detect when your original two instances are under heavy load. Trigger an automation script (e.g., AWS Lambda) to start the migration process.
• Notify clients to reconnect: The overloaded instances send a WebSocket message to all connected clients, instructing them to close their current connection and reconnect.
• Redirect new connections: When clients reconnect, the ALB can route them to idle new instances (you may need to temporarily adjust sticky session settings or use a short timeout during migration).
• Cache in-flight messages: Use Redis to temporarily store messages that were in transit during migration, so clients don’t miss data when they reconnect.

Whichever solution you choose, you need to handle graceful shutdowns during scaling down:

• Deregister from ALB first: Before terminating an instance, remove it from your ALB target group so it stops receiving new connections.
• Notify connected clients: The instance sends a "shutdown imminent" message to all WebSocket clients, prompting them to reconnect to another instance.
• Wait for cleanup: Give clients a window (e.g., 30 seconds) to reconnect, then close any remaining connections and terminate the instance.

Go with the Redis-based routing solution—it’s the most resilient for spiky workshop traffic, eliminates sticky session bottlenecks, and works seamlessly with auto-scaling. For AWS, pair it with Amazon ElastiCache for Redis to avoid managing your own Redis cluster.

内容的提问来源于stack exchange，提问作者Tobias