Spark Operator部署问题：spark-pi-driver运行但无Executor启动求助

按照相关指南部署Spark Operator后，提交Spark Pi应用时出现异常：spark-pi-driver Pod持续处于运行状态，但始终无法启动Executor Pod。已尝试更换无法访问的原镜像、调整RBAC权限配置，问题仍未解决。

已配置的RBAC规则 #

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: spark-role
rules:
- apiGroups: ["", "sparkoperator.k8s.io"]
  resources: ["events", "pods", "services", "configmaps", "sparkapplications/status", "scheduledsparkapplications", "sparkapplications"]
  verbs: ["*"]

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: spark-role-binding
subjects:
- kind: ServiceAccount
  name: spark
  namespace: spark-jobs
roleRef:
  kind: ClusterRole
  name: spark-role
  apiGroup: rbac.authorization.k8s.io

Spark Pi应用配置 #

apiVersion: "sparkoperator.k8s.io/v1beta2"
kind: SparkApplication
metadata:
  name: spark-pi
  namespace: spark-operator
spec:
  type: Scala
  mode: cluster
  image: "ghcr.io/googlecloudplatform/spark-operator:v1beta2-1.3.4-3.1.1"
  imagePullPolicy: Always
  mainClass: org.apache.spark.examples.SparkPi
  mainApplicationFile: "local:///opt/spark/examples/jars/spark-examples_2.12-3.1.1.jar"
  sparkVersion: "3.1.1"
  restartPolicy:
    type: Never
  volumes:
    - name: "test-volume"
      hostPath:
        path: "/tmp"
        type: Directory
  driver:
    cores: 1
    coreLimit: "1200m"
    memory: "512m"
    labels:
      version: 3.1.1
    serviceAccount: spark
    volumeMounts:
      - name: "test-volume"
        mountPath: "/tmp"
  executor:
    cores: 1
    instances: 1
    memory: "512m"
    labels:
      version: 3.1.1
    serviceAccount: spark
    volumeMounts:
      - name: "test-volume"
        mountPath: "/tmp"

Spark Pi Driver Pod日志 #

++ id -u
+ myuid=0
++ id -g
+ mygid=0
+ set +e
++ getent passwd 0
+ uidentry=root:x:0:0:root:/root:/bin/bash
+ set -e
+ echo 0
0
0
root:x:0:0:root:/root:/bin/bash
+ echo 0
+ echo root:x:0:0:root:/root:/bin/bash
+ [[ -z root:x:0:0:root:/root:/bin/bash ]]
+ exec /usr/bin/tini -s -- /usr/bin/spark-operator driver --properties-file /opt/spark/conf/spark.properties --class org.apache.spark.examples.SparkPi local:///opt/spark/examples/jars/spark-examples_2.12-3.1.1.jar

排查与解决建议 #

• 修正ServiceAccount命名空间匹配：RBAC绑定中的spark ServiceAccount属于spark-jobs命名空间，但SparkApplication部署在spark-operator命名空间。需在spark-operator命名空间创建同名ServiceAccount，或修改ClusterRoleBinding的subjects字段指定目标命名空间。
• 更换正确的Spark运行时镜像：当前使用的ghcr.io/googlecloudplatform/spark-operator是Operator自身镜像，而非Spark运行时镜像。需替换为包含Spark环境的镜像，例如ghcr.io/googlecloudplatform/spark:v3.1.1或国内镜像registry.cn-hangzhou.aliyuncs.com/apache/spark:3.1.1。
• 检查Spark Operator日志：查看spark-operator Pod的日志，排查是否存在应用调度、权限校验或镜像拉取相关错误。
• 验证资源配额：确认spark-operator命名空间有足够的CPU、内存配额，避免Executor因资源不足无法调度。
• 测试RBAC权限有效性：执行以下命令验证ServiceAccount权限：

  kubectl auth can-i create pods --as=system:serviceaccount:spark-operator:spark
  kubectl auth can-i create services --as=system:serviceaccount:spark-operator:spark
  

• 移除hostPath卷测试：先删除配置中的volumes和volumeMounts字段，排除宿主目录权限或节点一致性问题对应用的影响。

内容的提问来源于stack exchange，提问作者Dmitriy