Cloud Build访问私有GKE遇tcp 10.10.2.2:443超时问题求助
排查Cloud Build私有池访问私有GKE集群超时问题
已完成Cloud Build私有池访问私有GKE集群的配置,但部署或执行测试时出现以下错误:
Unable to connect to the server: dial tcp 10.10.2.2:443: i/o timeout ERROR ERROR: build step 0 "gcr.io/cloud-builders/kubectl" failed: step exited with non-zero status: 1
VPN隧道已全部建立,且GKE集群已将192.168.0.0/20加入授权网络,以下是配置详情:
VPN隧道列表
# gcloud compute vpn-tunnels list NAME REGION GATEWAY PEER_ADDRESS ha-vpn-tunnel1 us-east1 ha-vpn-1 35.242.12.184 ha-vpn-tunnel2 us-east1 ha-vpn-1 35.220.15.95 ha-vpn-tunnel3 us-east1 ha-vpn-2 35.242.6.151 ha-vpn-tunnel4 us-east1 ha-vpn-2 35.220.14.211
VPN网关列表
# gcloud compute vpn-gateways list NAME INTERFACE0 INTERFACE1 NETWORK REGION ha-vpn-1 35.242.6.151 35.220.14.211 demo-vpc us-east1 ha-vpn-2 35.242.12.184 35.220.15.95 private-pool-peering-vpc us-east1
路由器列表
# gcloud compute routers list NAME REGION NETWORK ha-vpn-router1 us-east1 demo-vpc ha-vpn-router2 us-east1 private-pool-peering-vpc my-router us-east1 demo-vpc
ha-vpn-router1 详情(demo-vpc)
# gcloud compute routers describe ha-vpn-router1 --region us-east1 bgp: advertiseMode: DEFAULT asn: 64515 keepaliveInterval: 20 bgpPeers: - advertiseMode: CUSTOM advertisedIpRanges: - description: '' range: 192.168.0.0/20 advertisedRoutePriority: 100 bfd: minReceiveInterval: 1000 minTransmitInterval: 1000 multiplier: 5 sessionInitializationMode: DISABLED enable: 'TRUE' enableIpv6: false interfaceName: router1-interface2 ipAddress: 169.254.1.2 name: router1-peer2 peerAsn: 64516 peerIpAddress: 169.254.1.1 - advertiseMode: CUSTOM advertisedIpRanges: - description: '' range: 192.168.0.0/20 advertisedRoutePriority: 100 bfd: minReceiveInterval: 1000 minTransmitInterval: 1000 multiplier: 5 sessionInitializationMode: DISABLED enable: 'TRUE' enableIpv6: false interfaceName: router1-interface1 ipAddress: 169.254.0.1 name: router1-peer1 peerAsn: 64516 peerIpAddress: 169.254.0.2 creationTimestamp: '2023-01-03T16:55:55.711-08:00' id: '2186172510416498804' interfaces: - ipRange: 169.254.1.2/30 linkedVpnTunnel: https://www.googleapis.com/compute/v1/projects/fakeproject/regions/us-east1/vpnTunnels/ha-vpn-tunnel2 name: router1-interface2 - ipRange: 169.254.0.1/30 linkedVpnTunnel: https://www.googleapis.com/compute/v1/projects/fakeproject/regions/us-east1/vpnTunnels/ha-vpn-tunnel1 name: router1-interface1 kind: compute#router name: ha-vpn-router1 network: https://www.googleapis.com/compute/v1/projects/fakeproject/global/networks/demo-vpc region: https://www.googleapis.com/compute/v1/projects/fakeproject/regions/us-east1 selfLink: https://www.googleapis.com/compute/v1/projects/fakeproject/regions/us-east1/routers/ha-vpn-router1
ha-vpn-router2 详情(private-pool-peering-vpc)
# gcloud compute routers describe ha-vpn-router2 --region us-east1 bgp: advertiseMode: DEFAULT asn: 64516 keepaliveInterval: 20 bgpPeers: - advertiseMode: CUSTOM advertisedIpRanges: - description: '' range: 10.10.2.0/28 advertisedRoutePriority: 100 bfd: minReceiveInterval: 1000 minTransmitInterval: 1000 multiplier: 5 sessionInitializationMode: DISABLED enable: 'TRUE' enableIpv6: false interfaceName: router2-interface1 ipAddress: 169.254.0.2 name: router2-peer1 peerAsn: 64515 peerIpAddress: 169.254.0.1 - advertiseMode: CUSTOM advertisedIpRanges: - description: '' range: 10.10.2.0/28 advertisedRoutePriority: 100 bfd: minReceiveInterval: 1000 minTransmitInterval: 1000 multiplier: 5 sessionInitializationMode: DISABLED enable: 'TRUE' enableIpv6: false interfaceName: router2-interface2 ipAddress: 169.254.1.1 name: router2-peer2 peerAsn: 64515 peerIpAddress: 169.254.1.2 creationTimestamp: '2023-01-03T16:56:06.404-08:00' id: '6144519957886103625' interfaces: - ipRange: 169.254.0.2/30 linkedVpnTunnel: https://www.googleapis.com/compute/v1/projects/fakeproject/regions/us-east1/vpnTunnels/ha-vpn-tunnel3 name: router2-interface1 - ipRange: 169.254.1.1/30 linkedVpnTunnel: https://www.googleapis.com/compute/v1/projects/fakeproject/regions/us-east1/vpnTunnels/ha-vpn-tunnel4 name: router2-interface2 kind: compute#router name: ha-vpn-router2 network: https://www.googleapis.com/compute/v1/projects/fakeproject/global/networks/private-pool-peering-vpc region: https://www.googleapis.com/compute/v1/projects/fakeproject/regions/us-east1 selfLink: https://www.googleapis.com/compute/v1/projects/fakeproject/regions/us-east1/routers/ha-vpn-router2
更新记录(2022-01-04)
修改cloudbuild.yaml执行kubectl config view,结果如下:
Running: kubectl config view apiVersion: v1 clusters: - cluster: certificate-authority-data: DATA+OMITTED server: https://10.10.2.2 name: gke_fake-project_us-east1_demo-gke contexts: - context: cluster: gke_fake-project_us-east1_demo-gke user: gke_fake-project_us-east1_demo-gke name: gke_fake-project_us-east1_demo-gke current-context: gke_fake-project_us-east1_demo-gke kind: Config preferences: {} users: - name: gke_fake-project_us-east1_demo-gke user: auth-provider: config: access-token: xxx cmd-args: config config-helper --format=json cmd-path: /builder/google-cloud-sdk/bin/gcloud expiry: "2023-01-05T03:17:33Z" expiry-key: '{.credential.token_expiry}' token-key: '{.credential.access_token}' name: gcp PUSH DONE
GKE集群授权网络配置:
gcloud container clusters describe demo-gke \ > --format="value(masterAuthorizedNetworksConfig)"\ > --region=us-east1 cidrBlocks=[{'cidrBlock': '10.0.0.0/8', 'displayName': 'internal_10'}, {'cidrBlock': '192.168.0.0/16', 'displayName': 'internal_192'}, {'cidrBlock': '172.16.0.0/12', 'displayName': 'internal_172'}];enabled=True
排查思路
- 验证BGP对等体状态:执行
gcloud compute routers get-status ha-vpn-router1 --region us-east1和gcloud compute routers get-status ha-vpn-router2 --region us-east1,确认BGP会话已建立,且双方已正确接收对方的路由(10.10.2.0/28需出现在ha-vpn-router1的路由表,192.168.0.0/20需出现在ha-vpn-router2的路由表)。 - 检查防火墙规则:
- 确认demo-vpc的防火墙规则允许来自192.168.0.0/20的流量访问GKE master的443端口(协议TCP)。
- 确认private-pool-peering-vpc的防火墙规则允许Cloud Build私有池实例发起对10.10.2.2:443的出站请求。
- 核对Cloud Build私有池IP范围:确认私有池使用的IP段确实属于192.168.0.0/20,避免授权网络范围不匹配。
- 测试双向连通性:
- 在demo-vpc内创建临时VM,尝试ping 192.168.0.0/20内的Cloud Build私有池IP,确认隧道下行连通。
- 若private-pool-peering-vpc内有可操作资源,尝试ping 10.10.2.2,确认隧道上行连通。
- 确认GKE master路由发布:检查10.10.2.2所属的子网路由是否已通过ha-vpn-router2的BGP正确发布到private-pool-peering-vpc。
- 验证VPN隧道关联:确认ha-vpn-tunnel1/2关联的是ha-vpn-1(属于demo-vpc),ha-vpn-tunnel3/4关联的是ha-vpn-2(属于private-pool-peering-vpc),避免跨VPC关联错误。
内容的提问来源于stack exchange,提问作者Gary Turner
相关产品推荐
相关产品推荐

