使用kubeadm安装K8s 1.26.2后核心组件Pod处于CrashLoopBackOff状态的求助
看起来你在Debian 11(内核5.10.0/x86_64)上部署K8s 1.26时遇到了棘手的问题——所有核心组件Pod都陷入CrashLoopBackOff循环,连kubectl都无法连接API Server的6443端口。我来帮你梳理下关键的排查方向:
先聚焦最核心的etcd组件
从你提供的kubelet日志和crictl输出能看到,etcd已经重启了134次,它是整个K8s集群的存储核心,它启动失败的话,后续的apiserver、controller-manager等组件肯定都没法正常运行。你得先拿到etcd启动失败的具体错误日志:
crictl logs 5910c6fb6fe53
常见的etcd启动失败原因包括:
- 数据目录权限异常(默认路径是
/var/lib/etcd,可以用ls -ld /var/lib/etcd检查,权限应该归属etcd用户) - 端口被占用(etcd默认使用2379/2380端口,用
ss -tulpn | grep -E '2379|2380'排查) - 证书问题(kubeadm生成的etcd证书是否有效、路径是否配置正确)
- 磁盘空间不足(用
df -h /var/lib/etcd检查目录剩余空间)
检查主机内核与系统配置
K8s 1.26对Debian 11的5.10内核是兼容的,但需要确保必要的内核模块和参数已正确配置:
- 确认
overlay和br_netfilter模块已加载:
lsmod | grep -E 'overlay|br_netfilter'
如果没加载,手动加载并设置开机自启:
modprobe overlay modprobe br_netfilter echo -e "overlay\nbr_netfilter" >> /etc/modules-load.d/k8s.conf
- 验证网络相关sysctl参数是否正确:
sysctl net.bridge.bridge-nf-call-iptables net.bridge.bridge-nf-call-ip6tables net.ipv4.ip_forward
这三个参数都应该返回1,如果不是,修改配置并生效:
cat >> /etc/sysctl.d/k8s.conf <<EOF net.bridge.bridge-nf-call-iptables = 1 net.bridge.bridge-nf-call-ip6tables = 1 net.ipv4.ip_forward = 1 EOF sysctl --system
排查API Server启动失败原因
etcd恢复正常后,apiserver才有可能启动,你可以先查看apiserver的日志找线索:
crictl logs d4157ed07257b
另外检查6443端口是否被其他服务占用:
ss -tulpn | grep 6443
其他辅助排查点
- 检查kubelet本身状态:
systemctl status kubelet,确认kubelet是否正常运行,有没有报错信息 - 临时关闭AppArmor试试(Debian默认启用,可能会阻止容器正常运行):
systemctl stop apparmor,然后重启kubelet:systemctl restart kubelet - 检查主机磁盘IO是否正常:
iostat -x 1,etcd对磁盘IO性能有要求,IO过高也会导致启动失败
附上你提供的原始日志信息,方便后续排查参考:
kubelet日志片段
Mar 09 17:51:55 devnew0 kubelet[369024]: I0309 17:51:55.933659 369024 scope.go:115] "RemoveContainer" containerID="4ace1812ec7d981b55a51f422287499bdacf240e7c739d50872e6de1892fa7a2"
Mar 09 17:51:55 devnew0 kubelet[369024]: E0309 17:51:55.934416 369024 pod_workers.go:965] "Error syncing pod, skipping" err="failed to "StartContainer" for "etcd" with CrashLoopBackOff: "back-off 10s restarting failed container=etcd pod=etcd-devnew0_kube-system(419ed404bfcd79b143181759583871d1)"" pod="kube-system/etcd-devnew0" podUID=419ed404bfcd79b143181759583871d1
kubectl连接报错
E0309 18:06:07.103712 375368 memcache.go:238] couldn't get current server API group list: Get "https://162.105.85.70:6443/api?timeout=32s": dial tcp 162.105.85.70:6443: connect: connection refused
...
The connection to the server 162.105.85.70:6443 was refused - did you specify the right host or port?
crictl ps -a输出
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID POD dde80047d3aff 240e201d5b0d8 34 seconds ago Exited kube-controller-manager 131 fc8b51e8b4b44 kube-controller-manager-devnew0 d4157ed07257b 63d3239c3c159 2 minutes ago Exited kube-apiserver 128 96cad2187e463 kube-apiserver-devnew0 5910c6fb6fe53 fce326961ae2d 4 minutes ago Exited etcd 134 1fb01d0ecc4de etcd-devnew0 d635e40f9aade 6f64e7135a6ec 5 minutes ago Exited kube-proxy 114 c6c2597e5097a kube-proxy-2zmq7 e87d4f6d34e6d db8f409d9a5d7 5 minutes ago Exited kube-scheduler 143 0afe7b6d78151 kube-scheduler-devnew0
备注:内容来源于stack exchange,提问作者caiji233

