在Kubernetes中配置Loki时遇‘Top level config不允许’错误求解决

问题背景 #

部署Loki时收到以下错误：

Top level 'config' is not allowed. Most common configuration sections are exposed under the loki section. If you need to override the whole config, provide the configuration as a string that can contain template expressions under loki.config. Alternatively, you can provide the configuration as an external secret

用户当前的Ansible配置（main.yaml）如下：

---
- name: Add grafana helm repo
  kubernetes.core.helm_repository:
    name: grafana
    repo_url: "{{loki_repo_url}}"
  check_mode: no
  tags: loki


- name: "Create namespace {{ loki_namespace }}"
  kubernetes.core.k8s:
    state: present
    kubeconfig: "{{ kube_config }}"
    api_version: v1
    apply: yes
    definition:
      kind: Namespace
      metadata:
        name:  "{{ loki_namespace }}"
        annotations:
          scheduler.alpha.kubernetes.io/node-selector: agentpool=system
  tags: loki

- name: "Create caddy sidecar config {{ loki_namespace }}"
  kubernetes.core.k8s:
    kubeconfig: "{{kube_config}}"
    namespace: "{{ loki_namespace }}"
    state: present
    definition:
      apiVersion: v1
      kind: Secret
      metadata:
        name:   caddy-config
      type: Opaque
      stringData:
        Caddyfile : |
          {
           admin off
          }
          :{{ loki_sidecar_port }} {
                  reverse_proxy localhost:{{ loki_port }}
                  header -Server
                  basicauth {
                          {{ loki_user }} {{ loki_password }}
                  }
                  log {
                    format json
                    output stdout
                  }
          }
  tags: loki


- name: "Deploy loki chart version {{loki_chart_version}} on cluster: {{cluster_name}}"
  kubernetes.core.helm:
    atomic: yes
    name: loki
    chart_ref: grafana/loki
    chart_version: "{{loki_chart_version}}"
    release_namespace: "{{loki_namespace}}"
    create_namespace: yes
    kubeconfig: "{{kube_config}}"
    update_repo_cache: no
    wait: yes
    wait_timeout: 5m
    force: no
    state: present
    values:
      # Mandatory label due to Kyverno policy
      podAnnotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "{{ loki_port | string }}"

      terminationGracePeriodSeconds: 300
      service:
        port: "{{ loki_port }}"

      config:
        server:
          http_listen_port: "{{ loki_port }}"
        schema_config:
          configs:
            - from: 2021-09-01
              store: boltdb-shipper
              object_store: azure
              schema: v11
              index:
                prefix: index_
                period: 24h
        storage_config:
          boltdb_shipper:
            shared_store: azure
            active_index_directory: /data/loki/boltdb-shipper-active
            cache_location: /data/loki/boltdb-shipper-cache
            cache_ttl: 24h
          azure:
            container_name: "{{ az_container_name }}"
            account_name: "{{ az_account_name }}"
            account_key: "{{ az_access_key }}"
            request_timeout: 5m
          filesystem:
            directory: /data/loki/chunks
        
        chunk_store_config:
          max_look_back_period: 4464h
        table_manager:
          retention_deletes_enabled: true
          retention_period: 4464h
      extraContainers:
        - name: caddy-sidecar
          image: "docker.io/library/caddy:{{ caddy_version }}"
          args:
            - "caddy"
            - "run"
            - "-config=/etc/caddy/Caddyfile"
            - "-watch"
          ports:
            - name: http-proxy
              containerPort: "{{ loki_sidecar_port }}"
              protocol: TCP
          volumeMounts:
            - name: caddy-config
              mountPath: /etc/caddy
      extraVolumes:
        - name: caddy-config
          secret:
            secretName: caddy-config
      extraPorts:
      ## Additional ports to the loki services. Useful to expose extra container ports.
        - port: "{{ loki_sidecar_port }}"
          protocol: TCP
          name: http-proxy
          targetPort: http-proxy
  tags: loki
  
- name: "Create ingress: {{ loki_namespace }}"
  tags: loki
  kubernetes.core.k8s:
    kubeconfig: "{{kube_config}}"
    namespace: "{{ loki_namespace }}"
    state: present
    definition:
      apiVersion: networking.k8s.io/v1
      kind: Ingress
      #ingressClassName: application-gateway
      metadata:
        name: ingress-loki
        annotations:
          # use the shared ingress-nginx
          cert-manager.io/cluster-issuer: letsencrypt-prod
          kubernetes.io/ingress.class: azure/application-gateway
          appgw.ingress.kubernetes.io/ssl-redirect: "true"
          appgw.ingress.kubernetes.io/health-probe-status-codes: "200-399, 401"
      spec:
        rules:
          - host: "loki.{{ base_domain }}"
            http:
              paths:
                - path: /
                  pathType: Prefix
                  backend:
                    service:
                      name: loki
                      port:
                        number: "{{ loki_sidecar_port }}"
        tls:
          - hosts:
              - "loki.{{ base_domain }}"
            secretName: loki-ingress-cert
...

用户尝试将config放到loki下但未解决问题，请求修复配置。

错误原因 #

新版本Grafana Loki Helm chart要求：

• 完整的Loki配置必须嵌套在loki.config字段下
• 配置内容必须以多行字符串形式传入（而非直接的YAML对象）

你之前的尝试只做了层级嵌套，但没把配置转为字符串格式，所以仍然报错。

修复后的配置（仅展示修改后的Helm部署部分） #

- name: "Deploy loki chart version {{loki_chart_version}} on cluster: {{cluster_name}}"
  kubernetes.core.helm:
    atomic: yes
    name: loki
    chart_ref: grafana/loki
    chart_version: "{{loki_chart_version}}"
    release_namespace: "{{loki_namespace}}"
    create_namespace: yes
    kubeconfig: "{{kube_config}}"
    update_repo_cache: no
    wait: yes
    wait_timeout: 5m
    force: no
    state: present
    values:
      # Mandatory label due to Kyverno policy
      podAnnotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "{{ loki_port | string }}"

      terminationGracePeriodSeconds: 300
      service:
        port: "{{ loki_port }}"

      # 核心修改：将原config转为loki.config下的多行字符串
      loki:
        config: |
          server:
            http_listen_port: {{ loki_port }}
          schema_config:
            configs:
              - from: 2021-09-01
                store: boltdb-shipper
                object_store: azure
                schema: v11
                index:
                  prefix: index_
                  period: 24h
          storage_config:
            boltdb_shipper:
              shared_store: azure
              active_index_directory: /data/loki/boltdb-shipper-active
              cache_location: /data/loki/boltdb-shipper-cache
              cache_ttl: 24h
            azure:
              container_name: "{{ az_container_name }}"
              account_name: "{{ az_account_name }}"
              account_key: "{{ az_access_key }}"
              request_timeout: 5m
            filesystem:
              directory: /data/loki/chunks
          
          chunk_store_config:
            max_look_back_period: 4464h
          table_manager:
            retention_deletes_enabled: true
            retention_period: 4464h
      extraContainers:
        - name: caddy-sidecar
          image: "docker.io/library/caddy:{{ caddy_version }}"
          args:
            - "caddy"
            - "run"
            - "-config=/etc/caddy/Caddyfile"
            - "-watch"
          ports:
            - name: http-proxy
              containerPort: "{{ loki_sidecar_port }}"
              protocol: TCP
          volumeMounts:
            - name: caddy-config
              mountPath: /etc/caddy
      extraVolumes:
        - name: caddy-config
          secret:
            secretName: caddy-config
      extraPorts:
      ## Additional ports to the loki services. Useful to expose extra container ports.
        - port: "{{ loki_sidecar_port }}"
          protocol: TCP
          name: http-proxy
          targetPort: http-proxy
  tags: loki

关键改动说明 #

1. 层级调整：将原顶层config块移动到loki字段下，形成loki.config结构
2. 格式转换：使用|将整个Loki配置转为多行字符串，完全符合chart的配置要求
3. 变量处理：字符串内的模板变量（如{{ loki_port }}）无需加引号，Helm会自动解析这些变量值

内容的提问来源于stack exchange，提问作者PrayingMantis