关于GKE MultiCluster Ingress适配on-premise硬件负载均衡器及Internal Load Balancer的技术咨询

Can GKE MultiCluster Ingress work with an Internal Load Balancer (ILB) with a static IP? #

Absolutely—though it’s not the default behavior, GKE MultiCluster Ingress (MCI) does support internal load balancers when configured explicitly. Here’s what you need to know:

• By default, MCI provisions an External Load Balancer (ELB), but you can override this with a Kubernetes annotation to force an internal LB.
• You can bind a pre-provisioned static internal IP address to the MCI, as long as the IP belongs to the same VPC (or a peered VPC via Partner Interconnect) as your GKE clusters.

Example MCI Configuration for Internal LB

apiVersion: networking.gke.io/v1
kind: MultiClusterIngress
metadata:
  name: my-internal-mci
  annotations:
    # Force internal load balancer type
    networking.gke.io/load-balancer-type: "Internal"
spec:
  template:
    spec:
      backend:
        serviceName: my-cross-cluster-service
        servicePort: 80
      # Bind to your pre-created static internal IP
      loadBalancerIP: "10.1.0.100"
  clusters:
  - name: cluster-us-central1
    zone: us-central1-a
  - name: cluster-europe-west1
    zone: europe-west1-b

Note: The static internal IP must be created in the same region as your MCI’s control plane, and must not be in use by another resource.

Compliance-Friendly Solution for Multi-Cluster Ingress via On-Prem Hardware LB #

Given your regulatory requirement that all external traffic enters through an on-prem hardware load balancer before routing to GCP via Partner Interconnect, here’s a step-by-step design:

1. Peer Your On-Prem Network with GCP VPC via Partner Interconnect

   • Ensure your on-prem hardware LB’s subnet is peered with your GCP VPC (or the shared VPC hosting your GKE clusters) using Partner Interconnect. This establishes a private, compliant routing path between on-prem and GCP.

2. Deploy Internal MultiCluster Ingress for GKE Clusters

   • Configure a single MCI (as shown above) to act as the entry point for all GKE cluster traffic in GCP. Use a static internal IP that’s reachable from your on-prem network via the peered connection.
   • Register all your GKE clusters’ backend services with the MCI’s backend pool—MCI will automatically distribute traffic across healthy pods in all connected clusters.

3. Configure On-Prem Hardware Load Balancer

   • Set up your on-prem hardware LB to accept external traffic (per your regulatory rules) and forward it to the static internal IP of your GCP MCI.
   • Ensure the LB’s forwarding rules match the ports/protocols used by your GKE services (e.g., 80/443 for HTTP/HTTPS).

4. Lock Down Network Access with Firewalls

   • On GCP, create firewall rules that only allow traffic from your on-prem hardware LB’s IP range to access the MCI’s internal IP and your GKE cluster nodes/pods. This prevents unauthorized traffic from reaching your workloads.
   • On your on-prem side, enforce rules that only allow external traffic to enter via the approved hardware LB, with no direct access to GCP resources.

5. Optional: Centralize Traffic Management

   • If you need more granular control (e.g., path-based routing across clusters), leverage MCI’s routing rules to direct specific traffic patterns to different cluster backends. For example:

     spec:
       template:
         spec:
           rules:
           - http:
               paths:
               - path: /api
                 pathType: Prefix
                 backend:
                   serviceName: api-service
                   servicePort: 80
               - path: /web
                 pathType: Prefix
                 backend:
                   serviceName: web-service
                   servicePort: 80
     

Key Validation Checks #

• Verify that traffic flows only through your on-prem hardware LB (use packet capture tools on both on-prem and GCP to confirm).
• Ensure the static internal IP of your MCI is not exposed to the public internet (GCP internal LBs are inherently private, but double-check VPC firewall rules).
• Confirm that all GKE cluster nodes are in subnets that are reachable via the Partner Interconnect peering.

内容的提问来源于stack exchange，提问作者Vikram Bailur