Nginx反向HTTP代理：所有upstream不可用时的行为探究

1. What happens when all Nginx upstream backends are unavailable? #

When every backend in your upstream pool is marked as unavailable—whether due to downtime, overload, or connection failures—here’s what Nginx does by default:

• Immediate client response: Nginx returns a 502 Bad Gateway error to the requesting client. This is standard behavior because it can’t find any healthy backend to proxy the request to.
• Backend retry logic: During the fail_timeout window (default is 10 seconds), Nginx won’t attempt to reconnect to backends marked as failed (after hitting the max_fails threshold, default is 1 failure). Once the fail_timeout expires, it will retry connecting to those backends again.
• Customization options:
  • If you’ve set proxy_next_upstream (e.g., proxy_next_upstream error timeout), Nginx will cycle through all available backends first—but if all are down, it still falls back to returning 502.
  • You can replace the default 502 page with a custom one using error_page 502 /maintenance.html;.

2. Behavior when all backends fail due to overload (within fail_timeout) #

Your specific scenario—where overload takes down the first backend, then cascades to others until none are available in the fail_timeout window—follows the core behavior above, but with key nuances:

• Cascading failure trigger: When the first backend hits its overload limit, Nginx marks it as failed after max_fails failed requests/connections. It then shifts traffic to the next backend, which quickly gets overwhelmed too, repeating the cycle.
• No fallback during fail_timeout: Once all backends are marked as failed, Nginx will reject all incoming requests with 502 until the fail_timeout period ends. During this time, it won’t probe backends to check if they’ve recovered.
• Mitigation tips:
  • Increase max_fails to avoid marking backends as failed too easily (e.g., server backend1.example.com max_fails=3;).
  • Shorten fail_timeout to reduce the window where no backends are probed (e.g., fail_timeout=5s;).
  • Tune timeout settings (proxy_connect_timeout, proxy_read_timeout) to match your backend’s expected response time, so Nginx doesn’t incorrectly mark a slow-but-healthy backend as failed.

Testing & Improving Nginx Throughput #

If you’re unsure how to validate your configuration and fix throughput issues, try these practical steps:

• Load testing tools:
  Use wrk (a modern, efficient load tester) to simulate realistic traffic:

  wrk -t4 -c200 -d60s http://your-nginx-endpoint.com/
  

  This uses 4 threads, 200 concurrent connections, and runs for 60 seconds—output includes requests per second (throughput), latency percentiles, and error rates to pinpoint bottlenecks.
• Enable Nginx status monitoring:
  Turn on the stub_status module to track real-time metrics:

  location /nginx_status {
      stub_status on;
      allow 127.0.0.1; # Restrict access to your internal IP
      deny all;
  }
  

  Visiting this endpoint shows active connections, total requests, and rejected connections—critical for spotting issues like full connection queues.
• Optimize upstream connections:
  Use keepalive connections to reduce backend connection overhead:

  upstream backend_pool {
      server backend1.example.com;
      server backend2.example.com;
      keepalive 64; # Maintain 64 persistent connections per backend
  }
  

  Also, adjust worker_connections in your nginx.conf to ensure Nginx can handle your expected concurrent load (default is 1024, but can be increased based on your server’s resources).

内容的提问来源于stack exchange，提问作者Danielle M.