安装spark-on-k8s-operator时配置DNS以启用Webhook求助

我已在spark-on-k8s-operator项目仓库提交了Issue，现向有相关经验的开发者求助。

我的K8s集群存在网络限制，必须为spark-on-k8s-operator配置自定义的dnsConfig和dnsPolicy。例如需要在Helm Chart的values.yaml中添加以下配置：

dnsConfig:
  nameservers:
  - XXX.XX.X.XXX
dnsPolicy: ClusterFirstWithHostNet

这个配置是必需的——如果不为集群中的Pod设置dnsPolicy: ClusterFirstWithHostNet，Webhook初始化阶段将无法解析kubernetes.default.svc域名，导致init-webhook和clean-up-webhook均报错。

我必须启用Webhook，才能让SparkApplication中driver和executor的自定义dnsConfig、dnsPolicy生效，但目前配置未达到预期效果。

我已经尝试过以下方案，但都未能解决问题：

• 部署后使用kubectl replace -f webhook.yaml --force命令编辑Webhook Job（Webhook Job内容如下）
• 编辑spark-operator的Deployment

Webhook Job的配置内容：

apiVersion: batch/v1
kind: Job
metadata:
  annotations:
    helm.sh/hook: pre-install, pre-upgrade
    helm.sh/hook-weight: "50"
  creationTimestamp: "2024-03-02T14:27:44Z"
  generation: 1
  labels:
    app.kubernetes.io/instance: spark
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: spark-operator
    app.kubernetes.io/version: v1beta2-1.3.8-3.1.1
    helm.sh/chart: spark-operator-1.1.27
  name: spark-spark-operator-webhook-init
  namespace: spark-operator
  resourceVersion: "8042824"
spec:
  backoffLimit: 6
  completionMode: NonIndexed
  completions: 1
  parallelism: 1
  suspend: false
  template:
    metadata:
    spec:
      containers:
      - command:
        - /usr/bin/gencerts.sh
        - -n
        - spark-operator
        - -s
        - spark-spark-operator-webhook
        - -r
        - spark-spark-operator-webhook-certs
        - -p
        image: ghcr.io/googlecloudplatform/spark-operator:v1beta2-1.3.8-3.1.1
        imagePullPolicy: IfNotPresent
        name: main
        resources: {}
        securityContext: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirstWithHostNet
      dnsConfig:
        nameservers:
        - XXX.XXX.X.X
      restartPolicy: OnFailure
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: spark-spark-operator
      serviceAccountName: spark-spark-operator
      terminationGracePeriodSeconds: 30

内容的提问来源于stack exchange，提问作者Paulo Henrique Zen