在OpenShift上配置Nginx代理后，被代理服务间歇性出现502错误

Let’s dig into why you’re seeing intermittent 502 Bad Gateway errors when proxying to your apimanager_svc in OpenShift. Since only this HTTPS-backed service is affected (and it’s a 50/50 split), here are the most likely causes and actionable fixes to try:

1. Missing SSL/TLS Handshake Configuration (Top Suspect) #

Your Nginx config proxies to an HTTPS backend for /apimanager, but you’re missing critical directives to handle the TLS handshake between Nginx and apimanager_svc. Intermittent failures here often stem from:

• Unverified backend SSL certificates (e.g., self-signed or cluster-internal CA that Nginx doesn’t trust)
• TLS handshake timeouts if the backend is slow to respond
• Missing SNI (Server Name Indication) causing the backend to reject connections

Fix Steps: #

Add these directives to your /apimanager location block to stabilize the TLS connection:

location /apimanager {
    set $upstreamapi https://apimanager_svc.<namespace>.svc.cluster.local:10666/api/v1/;
    proxy_pass $upstreamapi$request_uri;
    
    # Add SSL proxy config
    proxy_ssl_server_name on; # Send SNI to backend
    proxy_ssl_verify off; # Temporarily disable verification to test (replace with trusted CA in production)
    # If using cluster-internal CA, uncomment and mount the CA cert:
    # proxy_ssl_trusted_certificate /etc/ssl/certs/cluster-ca.crt;
    # proxy_ssl_verify on;
    
    # Add timeout settings to avoid handshake timeouts
    proxy_connect_timeout 10s;
    proxy_send_timeout 10s;
    proxy_read_timeout 10s;
}

2. Backend Connection Pool Exhaustion #

With worker_processes 1 and default connection settings, Nginx might run out of available connections to apimanager_svc if the backend has slow response times or high traffic. This leads to intermittent 502s when no connections are free.

Fix Steps: #

• Update the worker_processes to use all available CPU cores in your pod:

  worker_processes auto;
  

• Use an upstream block to enable keepalive connections to the backend (more efficient than per-request connections):

  http {
      # ... existing config ...
      
      upstream apimanager_backend {
          server apimanager_svc.<namespace>.svc.cluster.local:10666;
          keepalive 32; # Maintain 32 persistent connections to the backend
      }
      
      server {
          # ... existing server config ...
          
          location /apimanager {
              proxy_pass https://apimanager_backend/api/v1/;
              proxy_http_version 1.1;
              proxy_set_header Connection ""; # Disable connection close header for keepalive
              
              # Add SSL and timeout directives from Step 1 here
          }
      }
  }
  

3. DNS Resolution Flakiness #

Even with your resolver configured, intermittent DNS failures can cause Nginx to fail to resolve apimanager_svc occasionally. OpenShift’s internal DNS is reliable, but adding extra resilience helps.

Fix Steps: #

Add a resolver timeout to your http block to limit how long Nginx waits for a DNS response:

http {
    resolver dns-default.openshift-dns.svc.cluster.local valid=10s;
    resolver_timeout 5s; # Add this line
    # ... existing config ...
}

4. Backend Service Instability #

Don’t rule out issues with the apimanager_svc itself. Since both services use Beego v2, check if the backend pods are experiencing:

• Intermittent crashes or restarts
• High CPU/memory usage
• Connection limits being hit

Debug Commands (Run in OpenShift CLI): #

# Check recent logs for apimanager pods
oc logs -l app=apimanager_svc --tail=100 --follow

# Check if service endpoints are healthy
oc describe svc apimanager_svc

# Check pod resource usage
oc top pods -l app=apimanager_svc

If pods are flapping or endpoints are marked as not ready, that’s the root cause—fix the backend service first.

内容的提问来源于stack exchange，提问作者SoumitraM