EKS中Network Load Balancer暴露内部IP，如何实现公网访问？

我在EKS中创建了一个LoadBalancer服务，但它暴露的是内部IP地址。我不想使用ALB，希望使用NLB并在Pod中终止连接。Pod运行在Fargate节点上，请问还需要进行哪些操作才能将该服务暴露至公网？

服务定义 #

apiVersion: v1
kind: Service
metadata:
  name: echo-server
  namespace: dev-echo-server
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: "nlb-ip"
spec:
  selector:
    app: echo-server
  ports:
    - protocol: TCP
      port: 8080
      targetPort: http-8080
  type: LoadBalancer
  sessionAffinity: None
  externalTrafficPolicy: Cluster
  ipFamilies:
    - IPv4
  ipFamilyPolicy: SingleStack
  allocateLoadBalancerNodePorts: false
  loadBalancerClass: service.k8s.aws/nlb
  internalTrafficPolicy: Cluster

当前状态 #

外部IP属于内部网段，无法从公网访问：

$ kubectl get svc
NAME          TYPE           CLUSTER-IP      EXTERNAL-IP                                                                     PORT(S)    AGE
echo-server   LoadBalancer   172.20.246.19   k8s-devechos-echoserv-930c12fb8d-cd88a6c85735d503.elb.eu-west-1.amazonaws.com   8080/TCP   6m36s

$ host k8s-devechos-echoserv-930c12fb8d-cd88a6c85735d503.elb.eu-west-1.amazonaws.com
k8s-devechos-echoserv-930c12fb8d-cd88a6c85735d503.elb.eu-west-1.amazonaws.com has address 10.0.16.198

补充信息 #

VPC的Terraform定义 #

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 4.0"

  name = local.name
  cidr = local.vpc_cidr

  azs             = local.azs
  private_subnets = [for k, v in local.azs : cidrsubnet(local.vpc_cidr, 4, k)]
  public_subnets  = [for k, v in local.azs : cidrsubnet(local.vpc_cidr, 8, k + 48)]
  intra_subnets   = [for k, v in local.azs : cidrsubnet(local.vpc_cidr, 8, k + 52)]

  enable_nat_gateway = true
  single_nat_gateway = true

  public_subnet_tags = {
    "kubernetes.io/role/elb" = 1
  }

  private_subnet_tags = {
    "kubernetes.io/role/internal-elb" = 1
  }

  tags = local.tags
}

负载均衡器描述 #

{
    "LoadBalancers": [
        {
            "LoadBalancerArn": "arn:aws:elasticloadbalancing:eu-west-1:493334834594:loadbalancer/net/k8s-devechos-echoserv-930c12fb8d/cd88a6c85735d503",
            "DNSName": "k8s-devechos-echoserv-930c12fb8d-cd88a6c85735d503.elb.eu-west-1.amazonaws.com",
            "CanonicalHostedZoneId": "Z2IFOLAFXWLO4F",
            "CreatedTime": "2023-08-14T12:34:50.645000+00:00",
            "LoadBalancerName": "k8s-devechos-echoserv-930c12fb8d",
            "Scheme": "internal",
            "VpcId": "vpc-032abfa8b12ae2a87",
            "State": {
                "Code": "active"
            },
            "Type": "network",
            "AvailabilityZones": [
                {
                    "ZoneName": "eu-west-1a",
                    "SubnetId": "subnet-08a705830583829fc",
                    "LoadBalancerAddresses": []
                },
                {
                    "ZoneName": "eu-west-1b",
                    "SubnetId": "subnet-08e3697bb06aae5e7",
                    "LoadBalancerAddresses": []
                },
                {
                    "ZoneName": "eu-west-1c",
                    "SubnetId": "subnet-0b7c6812480cb628f",
                    "LoadBalancerAddresses": []
                }
            ],
            "IpAddressType": "ipv4"
        }
    ]
}

核心问题 #

从负载均衡器描述可以明确，当前NLB的Scheme为internal，且关联的是私有子网，这是它只能提供内部IP、无法公网访问的直接原因。

修复步骤 #

1. 修改Service配置，添加公网NLB注解 #

在Service的metadata.annotations中新增一行，明确指定NLB为面向公网的类型：

service.beta.kubernetes.io/aws-load-balancer-scheme: "internet-facing"

修改后的完整Service配置如下：

apiVersion: v1
kind: Service
metadata:
  name: echo-server
  namespace: dev-echo-server
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: "nlb-ip"
    service.beta.kubernetes.io/aws-load-balancer-scheme: "internet-facing"
spec:
  selector:
    app: echo-server
  ports:
    - protocol: TCP
      port: 8080
      targetPort: http-8080
  type: LoadBalancer
  sessionAffinity: None
  externalTrafficPolicy: Cluster
  ipFamilies:
    - IPv4
  ipFamilyPolicy: SingleStack
  allocateLoadBalancerNodePorts: false
  loadBalancerClass: service.k8s.aws/nlb
  internalTrafficPolicy: Cluster

2. 应用更新后的配置 #

执行命令将修改后的Service配置应用到集群：

kubectl apply -f <你的Service配置文件路径>

3. 验证结果 #

等待1-2分钟让AWS重新创建NLB，之后通过以下步骤验证：

• 查看Service的外部DNS：kubectl get svc echo-server -n dev-echo-server
• 解析DNS确认公网IP：host <上述命令得到的EXTERNAL-IP值>
• 检查NLB的Scheme是否变为internet-facing：aws elbv2 describe-load-balancers --load-balancer-arn <负载均衡器ARN>

说明 #

你的VPC已正确为公网子网打上kubernetes.io/role/elb:1标签，修改注解后，AWS Load Balancer Controller会自动将NLB部署到公网子网并生成公网可访问的IP。另外，Fargate环境下使用nlb-ip类型是正确的，无需改动其他现有配置。

内容的提问来源于stack exchange，提问作者gbajson