基于AWS Load Balancer Controller实现公网NLB+内部Ingress架构

集群基础配置 #

• Kubernetes版本：1.22
• AWS负载均衡控制器版本：v2.4.1
• 节点类型：托管节点

需求背景 #

我了解到可通过NGINX Ingress Controller实现「公网NLB+内部Ingress控制器」的架构，但因需将AWS WAF与Ingress对应的Application Load Balancer集成，必须仅使用AWS Load Balancer Controller实现该架构。

当前采用NodePort服务搭配公网ALB类型Ingress的方案存在间歇性连接问题，这是已知问题。

已部署的公网NLB服务配置 #

service.yaml（Network Load Balancer）

apiVersion: v1
kind: Service
metadata:
  name: my-app-svc
  namespace: test
  annotations:
    # 后端使用HTTP协议
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: <证书ARN>
    # 仅在下方名为"https"的端口启用SSL
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "https"
    service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "300"
    service.beta.kubernetes.io/aws-load-balancer-ssl-negotiation-policy: "ELBSecurityPolicy-TLS-1-2-2017-01"
    # NLB相关注解
    service.beta.kubernetes.io/aws-load-balancer-type: external
    service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
    service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
spec:
  selector:
    run: my-app-backend-deployment
  ports:
  - name: https
    protocol: TCP
    port: 443
    targetPort: 7000
  - name: http
    protocol: TCP
    port: 80
    targetPort: 7000
  type: LoadBalancer

当前有问题的配置（NodePort后端+公网ALB Ingress） #

当前配置文件

---
apiVersion: v1
kind: Service
metadata:
  namespace: dev
  name: my-backend-svc
spec:
  ports:
  - name: http
    port: 80
    targetPort: 80
  type: NodePort
  selector:
    run: my-app-backend-deployment
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  namespace: dev
  name: my-ingress
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/certificate-arn: <我的证书IAM ARN>
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
    alb.ingress.kubernetes.io/ssl-policy: ELBSecurityPolicy-TLS-1-2-Ext-2018-06
    alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}'
    alb.ingress.kubernetes.io/connection-idle-timeout: "300"
   
spec:
  ingressClassName: alb
  rules:
    - host: my.web.host.fqdn
      http:
        paths:
          - backend:
              service:
                name: ssl-redirect
                port:
                  name: use-annotation
            path: /*
            pathType: ImplementationSpecific
          - backend:
              service:
                name: my-backend-svc
                port:
                  number: 80
            path: /*
            pathType: ImplementationSpecific

诉求 #

获取可规避间歇性连接问题的内部Ingress控制器配置。

内容的提问来源于stack exchange，提问作者Arani