为何Kubernetes中Cluster IP Service可从宿主机/节点访问?
按照Kubernetes的定义,Cluster IP是仅面向集群内部Pod提供访问的虚拟IP,但实际测试中,在安装K8s的Ubuntu宿主机/节点上却能成功ping通Cluster IP(如示例中的10.101.210.88),这背后的原因是什么?
测试数据
/root#kgs NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE default kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 16d kube-system kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 16d ricinfra service-tiller-ricxapp ClusterIP 10.98.94.194 <none> 44134/TCP 7d7h ricplt aux-entry ClusterIP 10.105.149.143 <none> 80/TCP,443/TCP 7d7h ricplt r4-influxdb-influxdb2 ClusterIP 10.110.14.243 <none> 80/TCP 7d7h ricplt r4-infrastructure-kong-proxy NodePort 10.107.12.178 <none> 32080:32080/TCP,32443:32443/TCP 7d7h ricplt r4-infrastructure-prometheus-alertmanager ClusterIP 10.104.86.76 <none> 80/TCP 7d7h ricplt r4-infrastructure-prometheus-server ClusterIP 10.102.224.176 <none> 80/TCP 7d7h ricplt service-ricplt-a1mediator-http ClusterIP 10.105.45.1 <none> 10000/TCP 7d7h ricplt service-ricplt-a1mediator-rmr ClusterIP 10.108.188.147 <none> 4561/TCP,4562/TCP 7d7h ricplt service-ricplt-alarmmanager-http ClusterIP 10.111.239.130 <none> 8080/TCP 7d7h ricplt service-ricplt-alarmmanager-rmr ClusterIP 10.106.30.195 <none> 4560/TCP,4561/TCP 7d7h ricplt service-ricplt-appmgr-http ClusterIP 10.110.110.91 <none> 8080/TCP 7d7h ricplt service-ricplt-appmgr-rmr ClusterIP 10.110.96.28 <none> 4561/TCP,4560/TCP 7d7h ricplt service-ricplt-dbaas-tcp ClusterIP None <none> 6379/TCP 7d7h ricplt service-ricplt-e2mgr-http ClusterIP 10.101.210.88 <none> 3800/TCP 7d7h ricplt service-ricplt-e2mgr-rmr ClusterIP 10.101.245.34 <none> 4561/TCP,3801/TCP 7d7h ricplt service-ricplt-e2term-prometheus-alpha ClusterIP 10.97.95.213 <none> 8088/TCP 7d7h ricplt service-ricplt-e2term-rmr-alpha ClusterIP 10.100.36.142 <none> 4561/TCP,38000/TCP 7d7h ricplt service-ricplt-e2term-sctp-alpha NodePort 10.108.215.136 <none> 36422:32222/SCTP 7d7h ricplt service-ricplt-o1mediator-http ClusterIP 10.96.196.67 <none> 9001/TCP,8080/TCP,3000/TCP 7d7h ricplt service-ricplt-o1mediator-tcp-netconf NodePort 10.104.237.252 <none> 830:30830/TCP 7d7h ricplt service-ricplt-rtmgr-http ClusterIP 10.105.27.42 <none> 3800/TCP 7d7h ricplt service-ricplt-rtmgr-rmr ClusterIP 10.110.0.158 <none> 4561/TCP,4560/TCP 7d7h ricplt service-ricplt-submgr-http ClusterIP None <none> 3800/TCP 7d7h ricplt service-ricplt-submgr-rmr ClusterIP None <none> 4560/TCP,4561/TCP 7d7h ricplt service-ricplt-vespamgr-http ClusterIP 10.98.139.191 <none> 8080/TCP,9095/TCP 7d7h ricxapp aux-entry ClusterIP 10.99.152.66 <none> 80/TCP,443/TCP 7d7h ricxapp service-ricxapp-bouncer-xapp-http ClusterIP 10.99.222.68 <none> 8080/TCP 25h ricxapp service-ricxapp-bouncer-xapp-rmr ClusterIP 10.97.149.3 <none> 4560/TCP,4561/TCP 25h ricxapp service-ricxapp-example-http ClusterIP 10.101.14.166 <none> 8080/TCP 2d11h ricxapp service-ricxapp-example-rmr ClusterIP 10.97.129.85 <none> 4560/TCP,4561/TCP 2d11h /root# /root# /root#ping 10.101.210.88 --------> 该地址为Cluster IP,从宿主机/节点执行ping测试 PING 10.101.210.88 (10.101.210.88) 56(84) bytes of data. 64 bytes from 10.101.210.88: icmp_seq=1 ttl=64 time=0.061 ms 64 bytes from 10.101.210.88: icmp_seq=2 ttl=64 time=0.032 ms ^C --- 10.101.210.88 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1019ms rtt min/avg/max/mdev = 0.032/0.046/0.061/0.014 ms /root#
核心原因解析
Cluster IP的"仅集群内部访问"并非指排除宿主机节点,而是指仅能被Kubernetes集群网络覆盖的实体访问,而宿主机节点本身就是集群网络的一部分,具体原因如下:
kube-proxy在宿主机上维护的网络规则
kube-proxy会在每个节点上通过iptables或IPVS生成转发规则,这些规则不仅处理Pod到Cluster IP的流量,也会处理宿主机自身发出的流量。当你在宿主机ping Cluster IP时,流量会被kube-proxy的规则捕获,转发到对应Service后端的Pod,或者直接由节点网络栈响应(部分CNI插件会在节点上配置Cluster IP的本地路由)。节点属于集群网络平面
无论是flannel、calico还是其他CNI插件,都会将宿主机节点纳入集群的虚拟网络体系中。节点的网络栈能直接识别Cluster IP段的路由,流量无需离开集群网络即可到达目标Service。Cluster IP的本质是虚拟IP的转发入口
Cluster IP本身不是一个真实存在的网卡IP,而是kube-proxy通过网络规则构建的转发标识。只要流量能触达这些转发规则所在的节点(也就是集群内的任何节点),就能被正确转发,宿主机作为节点自然满足这个条件。
补充说明:如果想要限制宿主机访问Cluster IP,可以通过NetworkPolicy规则进行精细化控制,但默认情况下Kubernetes并不会禁止节点本身访问Cluster IP。
内容的提问来源于stack exchange,提问作者myquest5 sh

