在OpenShift上用Ansible或Helm测试Helm部署的K8s资源运行状态

以下代码针对指定Helm Release，覆盖Deployment、StatefulSet、PVC、Service等多种核心资源的健康状态校验，比单纯检查Deployment更全面：

- name: Validate all resources from AAP Helm release
  hosts: localhost
  gather_facts: false
  vars:
    helm_release_name: "aap-release"  # 替换为你的Helm Release名称
    aap_namespace: "aap"               # 替换为部署AAP的命名空间
    resource_types:
      - deployment
      - statefulset
      - service
      - persistentvolumeclaim
      - job

  tasks:
    - name: Fetch all resources for the Helm release
      shell: >
        oc get {{ item }} -n {{ aap_namespace }} -o json
        --selector app.kubernetes.io/instance={{ helm_release_name }}
      register: resource_output
      with_items: "{{ resource_types }}"
      failed_when: resource_output.rc != 0

    - name: Validate Deployment/StatefulSet readiness
      vars:
        resource: "{{ (item.stdout | from_json).items[0] }}"
      assert:
        that:
          - resource.status.readyReplicas is defined
          - resource.status.readyReplicas == resource.spec.replicas
        fail_msg: "{{ item.item }} {{ resource.metadata.name }} is not ready"
        success_msg: "{{ item.item }} {{ resource.metadata.name }} is ready"
      when: item.item in ['deployment', 'statefulset']
      loop: "{{ resource_output.results }}"

    - name: Validate PVC binding status
      vars:
        resource: "{{ (item.stdout | from_json).items[0] }}"
      assert:
        that:
          - resource.status.phase == "Bound"
        fail_msg: "PVC {{ resource.metadata.name }} is not bound"
        success_msg: "PVC {{ resource.metadata.name }} is bound"
      when: item.item == 'persistentvolumeclaim'
      loop: "{{ resource_output.results }}"

    - name: Validate Service availability
      vars:
        resource: "{{ (item.stdout | from_json).items[0] }}"
      assert:
        that:
          - resource.spec.clusterIP is defined or resource.spec.loadBalancerIP is defined
        fail_msg: "Service {{ resource.metadata.name }} has no valid IP"
        success_msg: "Service {{ resource.metadata.name }} is available"
      when: item.item == 'service'
      loop: "{{ resource_output.results }}"

    - name: Validate Job completion status
      vars:
        resource: "{{ (item.stdout | from_json).items[0] }}"
      assert:
        that:
          - resource.status.succeeded is defined
          - resource.status.succeeded >= 1
        fail_msg: "Job {{ resource.metadata.name }} did not complete successfully"
        success_msg: "Job {{ resource.metadata.name }} completed successfully"
      when: item.item == 'job'
      loop: "{{ resource_output.results }}"

说明： #

• 利用Helm自动添加的app.kubernetes.io/instance标签过滤资源，确保只校验当前Release部署的内容
• 可根据AAP实际部署的资源类型，调整resource_types列表（比如添加ingress或daemonset）
• 针对每种资源类型的状态做了针对性校验，确保资源处于健康可用状态

Helm Test通过在Chart中定义测试Pod，自动执行健康检查逻辑。以下是针对AAP组件的测试示例：

1. 在Helm Chart中添加测试文件 #

在Chart的tests目录下创建两个测试文件：

tests/pah-healthcheck.yaml（Private Automation Hub健康检查）

apiVersion: v1
kind: Pod
metadata:
  name: "{{ .Release.Name }}-pah-healthcheck"
  annotations:
    "helm.sh/hook": test
spec:
  containers:
    - name: pah-healthcheck
      image: registry.redhat.io/openshift4/ose-cli:latest
      command:
        - /bin/sh
        - -c
        - |
          curl -f http://{{ .Release.Name }}-pah-api.{{ .Release.Namespace }}.svc.cluster.local:8080/api/galaxy/v3/ && echo "PAH API is healthy"
  restartPolicy: Never

tests/controller-healthcheck.yaml（Automation Controller健康检查）

apiVersion: v1
kind: Pod
metadata:
  name: "{{ .Release.Name }}-controller-healthcheck"
  annotations:
    "helm.sh/hook": test
spec:
  containers:
    - name: controller-healthcheck
      image: registry.redhat.io/openshift4/ose-cli:latest
      command:
        - /bin/sh
        - -c
        - |
          curl -f http://{{ .Release.Name }}-controller-service.{{ .Release.Namespace }}.svc.cluster.local/api/ && echo "Automation Controller API is healthy"
  restartPolicy: Never

2. 运行Helm Test #

直接通过命令行执行：

helm test {{ helm_release_name }} -n {{ aap_namespace }}

3. Ansible调用Helm Test的代码示例 #

- name: Run Helm tests for AAP release
  hosts: localhost
  gather_facts: false
  vars:
    helm_release_name: "aap-release"
    aap_namespace: "aap"

  tasks:
    - name: Execute Helm test
      shell: helm test {{ helm_release_name }} -n {{ aap_namespace }}
      register: helm_test_result
      failed_when: helm_test_result.rc != 0

    - name: Show Helm test output
      debug:
        var: helm_test_result.stdout_lines

说明： #

• Helm Test Pod会在测试完成后自动删除（除非添加--keep-test-pods参数）
• 测试逻辑可根据AAP的实际暴露方式调整（比如启用Ingress后，可直接测试外部地址）
• --atomic参数用于Helm安装/升级阶段，确保失败时自动回滚，和Helm Test属于不同功能范畴，前者是部署容错，后者是部署后验证

内容的提问来源于stack exchange，提问作者I__sh