K8s集群Worker节点kube-proxy无法获取节点IP问题求助
K8s集群Worker节点kube-proxy无法获取节点IP问题求助
我现在维护着一个用Kubeadm搭建的K8s集群,包含3个Master节点和3个Worker节点,每个Master对应管理一个Worker。最近其中一个Worker节点出现了之前没遇到过的异常:kube-proxy Pod无法正确识别节点IP,持续报错无法连接到本地6443端口获取节点信息。
以下是相关日志信息:
I0226 10:03:50.904111 1066929 round_trippers.go:510] HTTP Trace: Dial to tcp:127.0.0.1:6443 succeed I0226 10:03:50.918311 1066929 round_trippers.go:553] GET https://127.0.0.1:6443/api/v1/namespaces/kube-system/pods/kube-proxy-ssss2 200 OK in 15 milliseconds I0226 10:03:50.918395 1066929 round_trippers.go:570] HTTP Statistics: DNSLookup 0 ms Dial 0 ms TLSHandshake 8 ms ServerProcessing 4 ms Duration 15 ms I0226 10:03:50.918447 1066929 round_trippers.go:577] Response Headers: I0226 10:03:50.918478 1066929 round_trippers.go:580] Date: Mon, 26 Feb 2024 10:03:50 GMT I0226 10:03:50.918529 1066929 round_trippers.go:580] Audit-Id: a7c16d6f-3cb0-48a8-aa73-e2e4fbffde64 I0226 10:03:50.918559 1066929 round_trippers.go:580] Cache-Control: no-cache, private I0226 10:03:50.918579 1066929 round_trippers.go:580] Content-Type: application/json I0226 10:03:50.918639 1066929 round_trippers.go:580] X-Kubernetes-Pf-Flowschema-Uid: 1cafbc8f-a46e-4e74-9804-03b72b0beb1c I0226 10:03:50.918661 1066929 round_trippers.go:580] X-Kubernetes-Pf-Prioritylevel-Uid: 447a939f-c32c-455a-bc49-1bc69b1544e4 I0226 10:03:50.918906 1066929 request.go:1171] Response Body: {"kind":"Pod","apiVersion":"v1","metadata":{"name":"kube-proxy-ssss2","generateName":"kube-proxy-","namespace":"kube-system","uid":"be28ae46-b701-4466-ade5-01032008209d","resourceVersion":"117095929","creationTimestamp":"2024-02-26T04:16:43Z","labels":{"controller-revision-hash":"6bc4695d8c","k8s-app":"kube-proxy","pod-template-generation":"1"},"ownerReferences":[{"apiVersion":"apps/v1","kind":"DaemonSet","name":"kube-proxy","uid":"15ecadb4-5d6c-4fb4-8f51-36dbb9c1a5a4","controller":true,"blockOwnerDeletion":true}],"managedFields":[{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2024-02-26T04:16:43Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:generateName":{}, I0226 10:03:50.947158 1066929 round_trippers.go:580] Audit-Id: fda7a277-2512-4d58-a1a8-7bc48e4b1752 I0226 10:03:50.947183 1066929 round_trippers.go:580] Cache-Control: no-cache, private I0226 10:03:50.947202 1066929 round_trippers.go:580] Content-Type: text/plain I0226 10:03:50.947222 1066929 round_trippers.go:580] Date: Mon, 26 Feb 2024 10:03:50 GMT E0226 04:16:43.854878 1 node.go:152] Failed to retrieve node info: Get "https://127.0.0.1:6443/api/v1/nodes/k8s-worker-03": dial tcp 127.0.0.1:6443: connect: connection refused E0226 04:16:45.010666 1 node.go:152] Failed to retrieve node info: Get "https://127.0.0.1:6443/api/v1/nodes/k8s-worker-03": dial tcp 127.0.0.1:6443: connect: connection refused E0226 04:16:47.300738 1 node.go:152] Failed to retrieve node info: Get "https://127.0.0.1:6443/api/v1/nodes/k8s-worker-03": dial tcp 127.0.0.1:6443: connect: connection refused E0226 04:16:51.658415 1 node.go:152] Failed to retrieve node info: Get "https://127.0.0.1:6443/api/v1/nodes/k8s-worker-03": dial tcp 127.0.0.1:6443: connect: connection refused E0226 04:17:01.247069 1 node.go:152] Failed to retrieve node info: Get "https://127.0.0.1:6443/api/v1/nodes/k8s-worker-03": dial tcp 127.0.0.1:6443: connect: connection refused E0226 04:17:19.267198 1 node.go:152] Failed to retrieve node info: Get "https://127.0.0.1:6443/api/v1/nodes/k8s-worker-03": dial tcp 127.0.0.1:6443: connect: connection refused I0226 04:17:19.267293 1 server.go:820] "Can't determine this node's IP, assuming 127.0.0.1; if this is incorrect, please set the --bind-address flag" I0226 04:17:19.267352 1 server_others.go:109] "Detected node IP" address="127.0.0.1" I0226 04:17:19.304643 1 server_others.go:248] "Using ipvs Proxier"
我已经尝试过重置该Worker节点并重新加入对应的Master节点,但问题依旧,kube-proxy仍然无法获取正确的节点IP。
从日志中可以看到,kube-proxy请求Pod自身的信息(https://127.0.0.1:6443/api/v1/namespaces/kube-system/pods/kube-proxy-ssss2)能正常返回200状态,但请求节点信息(https://127.0.0.1:6443/api/v1/nodes/k8s-worker-03)时却持续报connection refused。我猜测这可能是导致节点IP识别失败的根源,但不清楚具体原因是什么,希望能得到大家的帮助和排查思路。
备注:内容来源于stack exchange,提问作者user2643892
相关产品推荐
相关产品推荐

