Cloudflare+Amazon NLB+Ingress-Nginx架构下X-Forwarded-For头部异常问题求助

I'm having trouble getting the correct IP header values in my Kubernetes setup with the following proxy chain: Cloudflare → Amazon NLB → Ingress-nginx. The X-REAL-IP header shows the correct end-user IP, but X-FORWARDED-FOR is stuck on an internal NLB IP. Here's the full context:

Current Architecture #

• Cloudflare (CDN/edge proxy)
• Amazon Network Load Balancer (NLB)
• Ingress-nginx (Kubernetes ingress controller)

Ingress-Nginx Configuration #

config:
  use-forwarded-headers: "true"
  real-ip-header: "CF-Connecting-IP"
  forwarded-for-header: "CF-Connecting-IP"
  set-real-ip-from: "0.0.0.0/0"
  proxy-buffer-size: "16k"
  proxy-buffers-number: "8"

Observed Anomalies #

When checking request headers at the application level:

• REMOTE ADDR: 127.0.0.1
• X-FORWARDED-FOR: 10.0.102.38 (incorrect — this is an internal NLB IP)
• X-REAL-IP: xx.xxx.xxx.xxx (correct — matches the end user's IP from Cloudflare)

AWS NLB Configuration (Ingress-Nginx Service Annotations) #

service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp
service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: 3600
service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: xxxx
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: https
service.beta.kubernetes.io/aws-load-balancer-type: nlb

• ExternalTrafficPolicy: Local is enabled for both the Ingress-Nginx service and my application services.

Failed Attempt #

Enabling use-proxy-protocol: "true" in the Ingress-Nginx config breaks the application entirely, with this error:

2021-09-19 12:42:17 " while reading PROXY protocol, client: x.x.x.x, server: 0.0.0.0:80

I suspect this is incompatible with Cloudflare, since Cloudflare doesn't send PROXY protocol traffic to the NLB.

Solution #

Let's fix the X-Forwarded-For header by addressing the root causes in your configuration:

1. Override X-Forwarded-For Explicitly #

Your original config uses forwarded-for-header, but this can clash with the NLB adding its own IP to the header chain. Instead, explicitly set X-Forwarded-For to use Cloudflare's trusted IP header. Update your Ingress-Nginx config:

config:
  use-forwarded-headers: "true"
  real-ip-header: "CF-Connecting-IP"
  # Explicitly set X-Forwarded-For to the end user's IP from Cloudflare
  proxy-set-header: "X-Forwarded-For $http_cf_connecting_ip"
  # Restrict trusted IP ranges to Cloudflare's official list (security best practice)
  set-real-ip-from: "103.21.244.0/22"
  set-real-ip-from: "103.22.200.0/22"
  set-real-ip-from: "103.31.4.0/22"
  set-real-ip-from: "104.16.0.0/13"
  set-real-ip-from: "104.24.0.0/14"
  set-real-ip-from: "108.162.192.0/18"
  set-real-ip-from: "131.0.72.0/22"
  set-real-ip-from: "141.101.64.0/18"
  set-real-ip-from: "162.158.0.0/15"
  set-real-ip-from: "172.64.0.0/13"
  set-real-ip-from: "173.245.48.0/20"
  set-real-ip-from: "188.114.96.0/20"
  set-real-ip-from: "190.93.240.0/20"
  set-real-ip-from: "197.234.240.0/22"
  set-real-ip-from: "198.41.128.0/17"
  proxy-buffer-size: "16k"
  proxy-buffers-number: "8"

• This bypasses any intermediate IPs added by the NLB and directly uses the trusted CF-Connecting-IP for X-Forwarded-For.
• Restricting set-real-ip-from to Cloudflare's IP ranges prevents header spoofing.

2. Remove Proxy Protocol from NLB Annotations #

You've enabled proxy protocol on the NLB, but Cloudflare doesn't send PROXY protocol traffic — it uses standard HTTP/HTTPS headers. This mismatch is likely causing header mangling. Remove this annotation from your Ingress-Nginx service:

# Delete this line:
# service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"

3. Adjust ExternalTrafficPolicy for Application Services #

Keep ExternalTrafficPolicy: Local only on the Ingress-Nginx service. For your application services, set it to Cluster (unless you have a specific need for Local). This avoids unnecessary IP routing complications.

4. Test the Changes #

1. Restart your Ingress-Nginx controller pods to apply the new config.
2. Send a test request and check the X-Forwarded-For header at your application — it should now match the correct end-user IP from X-REAL-IP.

内容的提问来源于stack exchange，提问作者thecodeassassin