Airflow Kubernetes Executor执行任务时无法创建Pod问题求助

Let’s walk through systematic steps to diagnose why your Airflow tasks are stuck in "running" without creating pods, even after setting up RBAC and base components.

1. Check Airflow Scheduler Logs for Direct Errors #

The first place to look is the Airflow scheduler’s logs—they’ll almost always contain clues about why pod creation is failing. Run this command to fetch and follow the scheduler logs:

kubectl logs -n airflow-staging <your-scheduler-pod-name> --follow

Keep an eye out for keywords like kubernetes_executor, pod_creation, permission denied, or failed to create pod. Common issues here include missing RBAC permissions, invalid pod templates, or namespace mismatches.

2. Verify RBAC Permissions Are Actually Effective #

You’ve created a ClusterRole and ClusterRoleBinding, but let’s confirm the service account can actually create pods in your namespace. Run this authorization check:

kubectl auth can-i create pods --as=system:serviceaccount:airflow-staging:airflow -n airflow-staging

If the output is no, your RBAC setup has a problem. A few things to adjust:

• Double-check for typos in your ClusterRoleBinding (your current config looks correct, but it never hurts to verify).
• Airflow’s Kubernetes Executor needs more permissions than basic pod CRUD. Update your pods-permissions ClusterRole to include these critical verbs and resources:

  rules:
  - apiGroups: [""]
    resources: ["pods", "pods/exec", "pods/logs", "pods/attach"]
    verbs: ["get", "list", "watch", "create", "delete", "patch", "update"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["get", "list", "watch"]
  

  Airflow uses these to interact with pods (fetch logs, execute commands) and report task status back to the scheduler.

3. Validate Airflow Kubernetes Executor Configuration #

Double-check your values-staging.yaml to ensure critical Kubernetes Executor settings are correctly set:

• Confirm you’ve specified the executor type:

  executor: KubernetesExecutor
  

• Verify the Kubernetes namespace for task pods matches your deployment namespace:

  kubernetes:
    namespace: airflow-staging
  

• Ensure the scheduler and worker pods are using the correct service account:

  scheduler:
    serviceAccountName: airflow
  worker:
    serviceAccountName: airflow
  

• Check if pod creation is explicitly enabled (some charts include a flag for this):

  kubernetes:
    allowPodCreation: true
  

4. Check Kind Cluster-Specific Limitations #

Kind has a few quirks that can block pod creation:

• Image Pull Issues: If your task pod uses an image that’s not available locally or in a public registry, Kind might fail to pull it. If a pod attempt is logged, run kubectl describe pods -n airflow-staging to check for ImagePullBackOff errors. You can load local images into Kind with:

  kind load docker-image <your-image-name> --name <your-cluster-name>
  

• Resource Constraints: Kind nodes have limited default resources. Check if your task pods are requesting more CPU/memory than available:

  kubectl describe nodes
  

  Look at the Allocated resources section to see if resources are exhausted.

5. Inspect Airflow Task & Pod Template Configurations #

If your DAG uses custom pod templates, ensure they’re valid:

• Check for typos in the template YAML (e.g., invalid resource requests, missing image fields).
• Ensure the template doesn’t specify a different namespace that the service account doesn’t have access to.

内容的提问来源于stack exchange，提问作者neku_dev