Kubernetes集群节点/Pod无法通过ClusterIP访问kube-apiserver求助

Hey there, I totally get the frustration—manual Kubernetes setup is no walk in the park, especially when you’re just starting out and you’ve sunk a week into it. Let’s break down how to troubleshoot this connectivity issue step by step, since this is a common pain point with manual clusters.

1. Verify Kube-APIServer Service & Core Configuration #

First, let’s confirm the basics of your API server’s service and runtime state:

• Grab the API server’s ClusterIP details with this command:

  kubectl get svc kubernetes -n default
  

  You should see a ClusterIP type service (usually 10.96.0.1 by default) with port 443/TCP in the READY state.
• On your master node, check that the kube-apiserver process is running with the correct network flags:

  ps aux | grep kube-apiserver
  

  Confirm the --advertise-address matches your master node’s internal IP, and --service-cluster-ip-range aligns with your cluster’s service CIDR (default is 10.96.0.0/12—this must not overlap with your Pod CIDR).

2. Test Connectivity From Inside a CoreDNS/Kube-Dns Pod #

Next, we’ll test directly from the problematic Pod to isolate where the issue lies:

• Get the name of your CoreDNS/kube-dns Pod:

  kubectl get pods -n kube-system | grep -E 'coredns|kube-dns'
  

• Exec into the Pod and test API server reachability:

  kubectl exec -it <your-dns-pod-name> -n kube-system -- sh
  # Inside the Pod, test HTTPS connectivity (skip cert check with -k)
  curl -k https://<api-server-cluster-ip>:443
  # Also try pinging the ClusterIP to check basic network reachability
  ping <api-server-cluster-ip>
  

• While inside the Pod, check its DNS configuration to ensure it’s using the cluster DNS correctly:

  cat /etc/resolv.conf
  

  You should see a nameserver entry pointing to your kube-dns/CoreDNS ClusterIP, plus a search domain like cluster.local.

3. Check Kube-Proxy Health & Configuration #

Kube-proxy is responsible for routing traffic from ClusterIPs to backend services, so it’s critical it’s working:

• Verify kube-proxy is running on all nodes (master and workers):

  # For systemd-managed nodes
  systemctl status kube-proxy
  # Or check via Kubernetes
  kubectl get pods -n kube-system | grep kube-proxy
  

• If using iptables mode (default for 1.9.6), check that the correct DNAT rules exist for the API server ClusterIP:

  iptables-save | grep <api-server-cluster-ip>
  

  You should see rules under the KUBE-SERVICES chain that forward traffic to your master node’s API server IP and port (usually 6443).

4. Validate Cluster Network Routing & CNI Setup #

Manual clusters often fail due to misconfigured CNI or routing:

• Double-check that your service CIDR and Pod CIDR do not overlap. For example, if you’re using Flannel, your Pod CIDR might be 10.244.0.0/16—this should be completely separate from the service CIDR.
• On worker nodes, check the routing table to ensure traffic to the service CIDR is routed correctly:

  ip route
  

  You should see a route for your service CIDR pointing to the CNI’s virtual interface (e.g., flannel.1 for Flannel) or kube-proxy’s network setup.
• Confirm your CNI plugin (Flannel, Calico, etc.) is fully deployed and all Pods are running:

  kubectl get pods -n kube-system | grep -E 'flannel|calico'
  

5. Rule Out Firewall/SELinux Interference #

Ubuntu 16.04’s UFW or SELinux (though rare on Ubuntu) can block cluster traffic:

• Temporarily disable UFW to test if it’s the culprit:

  ufw disable
  

  Retest connectivity—if it works, you’ll need to add rules to allow cluster internal traffic (service CIDR, Pod CIDR, port 6443, etc.).
• Check SELinux status (Ubuntu defaults to disabled, but worth confirming):

  sestatus
  

  If it’s in enforcing mode, switch to permissive temporarily to rule it out:

  setenforce 0
  

Quick Reminders for K8s 1.9.6 Manual Setup #

Since you’re working with an older version, keep these common pitfalls in mind:

• CNI Version Compatibility: Use a CNI plugin version that matches K8s 1.9.6—for example, Flannel v0.10.0 works well with this release; newer versions may have compatibility issues.
• Kubelet DNS Flags: Ensure all nodes’ kubelet processes have --cluster-dns set to your kube-dns/CoreDNS ClusterIP and --cluster-domain=cluster.local in their startup arguments.
• API Server Auth: CoreDNS needs a valid service account to access the API server—confirm the kube-dns service account in the kube-system namespace exists and has the correct permissions.

I know this feels overwhelming right now, but manual setup is absolutely the best way to learn how Kubernetes ticks—every issue you fix teaches you something critical. Take it step by step, and let’s get this cluster up and running!

内容的提问来源于stack exchange，提问作者Roman T.