如何用单个NLB管理多微服务及API网关500错误排查

执行kubectl apply -f service.yaml部署Service时，API网关和已部署的集群之间出现连接错误，仅返回状态码500（InternalServerErrorException），无具体错误信息。

环境参数 #

CLUSTER_NAME: test-service
SECURITY_GROUP_IDS: sg-xxxxxxxxxx,sg-xxxxxxxxxxx
VPC_PRIVATE_SUBNETS: subnet-xxxxxxxxxxxx,subnet-xxxxxxxxxxxx,subnet-xxxxxxxxxxxx
VPC_PUBLIC_SUBNETS: subnet-xxxxxxxxxxxx,subnet-xxxxxxxxxxxx,subnet-xxxxxxxxxxxx
NLB_NAME: "nlb-microservice"
DEPLOYMENT_IMAGE_NAME: "eks-image"
PORT: 8080
TARGET_PORT: 8080
NODE_PORT: 30164

对应的service.yaml配置 #

---
apiVersion: v1
kind: Service
metadata:
  name: ${NLB_NAME}
  namespace: ${CLUSTER_NAME}
  labels:
    app: ${NLB_NAME}
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-name: ${NLB_NAME}
    service.beta.kubernetes.io/aws-load-balancer-security-groups: ${SECURITY_GROUP_IDS}
    service.beta.kubernetes.io/aws-load-balancer-scheme: "internet-facing"
    service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-protocol: "HTTP"
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-port: "${PORT}"
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-path: "/healthcheck"
    service.beta.kubernetes.io/aws-load-balancer-subnets: ${VPC_PRIVATE_SUBNETS},${VPC_PUBLIC_SUBNETS}
    service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: "instance"
    service.beta.kubernetes.io/aws-load-balancer-target-group-attributes: deregistration_delay.timeout_seconds=300,stickiness.enabled=false,proxy_protocol_v2.enabled=false,stickiness.type=source_ip,deregistration_delay.connection_termination.enabled=false,preserve_client_ip.enabled=true
spec:
  type: LoadBalancer
  selector:
    app: ${DEPLOYMENT_IMAGE_NAME}
  ports:
    - port: ${PORT}
      protocol: TCP
      targetPort: ${TARGET_PORT}
      nodePort: ${NODE_PORT}

---
apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata:
  name: ${NLB_NAME}-tgb
  namespace: ${CLUSTER_NAME}
  labels:
    app: ${NLB_NAME}
spec:
  targetGroupARN: ${TARGET_GROUP_ARN}
  serviceRef:
    name: ${NLB_NAME}
    port: ${PORT}
  targetType: instance
  nodeSelector:
    matchLabels:
      beta.kubernetes.io/instance-type: t2.small
      alpha.eksctl.io/cluster-name: ${CLUSTER_NAME}

预期架构 #

目标是实现单个负载均衡器通过目标组转发流量管理多个微服务，架构如下：

+-----------------+
                          |     Gateway     |
                          +--------+--------+
                                   |
                                   v
                          +--------+--------+
                          | Load Balancer   |
                          +--------+--------+
                                   |
          +------------------------+-------------------------+
          |                        |                         |
          v                        v                         v
 +--------+--------+      +--------+--------+       +--------+--------+
 | Cluster 1       |      | Cluster 2       |       | Cluster 3       |
 | +-------------+ |      | +-------------+ |       | +-------------+ |
 | | Microservice| |      | | Microservice| |       | | Microservice| |
 | |     A       | |      | |     B       | |       | |     C       | |
 | +-------------+ |      | +-------------+ |       | +-------------+ |
 +-----------------+      +-----------------+       +-----------------+

已尝试的操作 #

移除yaml中的service.beta.kubernetes.io/aws-load-balancer-name和service.beta.kubernetes.io/aws-load-balancer-security-groups两个注解，不指定现有NLB进行部署。该操作会为每个微服务集群创建新的NLB、目标组和安全组，创建VPC链接关联新NLB后连接返回状态码200，但不符合预期架构。

内容的提问来源于stack exchange，提问作者Herian Palencia