Helm部署Airflow日志已同步GCS但UI无法显示求助

问题背景 #

通过Helm部署最新版Airflow，已配置将日志推送至GCS，且确认日志已存入对应存储桶，但Airflow UI无法展示这些日志。

UI错误日志 #

*** No logs found in GCS; ti=%s <TaskInstance: xyz> *** Could not read served logs: [Errno -2] Name or service not known

现有values.yaml中GCS相关配置 #

# Airflow scheduler settings
scheduler:
  extraVolumeMounts:
    - name: google-cloud-key
      mountPath: /opt/airflow/secrets
  extraVolumes:
    - name: google-cloud-key
      secret:
        secretName: sc-key

# Airflow webserver settings
webserver:
  extraVolumeMounts:
    - name: google-cloud-key
      mountPath: /opt/airflow/secrets
  extraVolumes:
    - name: google-cloud-key
      secret:
        secretName: sc-key

workers:
  extraVolumeMounts:
    - name: google-cloud-key
      mountPath: /opt/airflow/secrets
  extraVolumes:
    - name: google-cloud-key
      secret:
        secretName: sc-key

# Airflow triggerer settings
triggerer:
  extraVolumeMounts:
    - name: google-cloud-key
      mountPath: /opt/airflow/secrets
  extraVolumes:
    - name: google-cloud-key
      secret:
        secretName: sc-key

config:
  logging:
    remote_logging: 'True'
    remote_base_log_folder: 'gs://airflow-bucket/logs/'
    remote_log_conn_id: 'sc-key'
    google_key_path:  "/opt/airflow/secrets/key.json"

connections:
  - id: sc-key
    type: google_cloud_platform
    description: my GCP connection
        extra: |
      {
        "extra__google_cloud_platform__key_path": "/opt/airflow/secrets/key.json",
        "extra__google_cloud_platform__num_retries": 5
      }
env:
  - name: GOOGLE_APPLICATION_CREDENTIALS
    value: "/opt/airflow/secrets/key.json"
  - name: AIRFLOW__LOGGING__REMOTE_LOGGING
    value: "True"
  - name: AIRFLOW__LOGGING__REMOTE_LOG_CONN_ID
    value: "sc-key"
  - name: AIRFLOW__CORE__REMOTE_BASE_LOG_FOLDER
    value: "gs://airflow-bucket/logs/"
  - name: AIRFLOW__CORE__GOOGLE_KEY_PATH
    value: "opt/airflow/secrets/key.json"

extraPipPackages:
  - apache-airflow[google]

配置问题排查结果 #

• 环境变量路径缺失前缀：AIRFLOW__CORE__GOOGLE_KEY_PATH的值少了开头的/，正确路径应为/opt/airflow/secrets/key.json，当前配置会导致路径解析失败。
• Connections配置缩进错误：extra字段缩进层级错误，需与id、type同级，否则连接配置无法被正确加载。修正后如下：

  connections:
    - id: sc-key
      type: google_cloud_platform
      description: my GCP connection
      extra: |
        {
          "extra__google_cloud_platform__key_path": "/opt/airflow/secrets/key.json",
          "extra__google_cloud_platform__num_retries": 5
        }
  

• 配置参数路径错误：环境变量中AIRFLOW__CORE__REMOTE_BASE_LOG_FOLDER命名错误，对应Airflow配置的正确路径应为AIRFLOW__LOGGING__REMOTE_BASE_LOG_FOLDER，与config.logging下的配置保持一致。
• 权限验证补充：虽然日志可写入GCS，但需确认webserver pod使用的服务账号拥有storage.objects.get权限，确保UI能读取存储桶内的日志文件。

内容的提问来源于stack exchange，提问作者Tushar Raichand