Ansible脚本异常：如何正确过滤无绑定的过期certkey

问题背景 #

现有两个数据列表：

1. expired列表（存储已过期证书信息）：

[
    {
        "cert": "help.abc.com.cer",
        "certkey": "help.abc.com-key",
        "daystoexpiration": 0,
        "key": "help.abc.com.key"
    },
    {
        "cert": "prod.abc.ca-2020.cer",
        "certkey": "prod.abc.ca",
        "daystoexpiration": 0,
        "key": "prod.abc.ca-2020.key"
    },
    {
        "cert": "ca-profile-service-prod.abc.com.cer",
        "certkey": "ca-profile-service-prod-cert",
        "daystoexpiration": 0,
        "key": "ca-profile-service-prod.abc.com.key"
    },
    {
        "cert": "eclosing.abc.com.cer",
        "certkey": "eclosing-cert",
        "daystoexpiration": 0,
        "key": "eclosing.abc.com.key"
    },
    {
        "cert": "merlin-ldp-stg.abc.com.cer",
        "certkey": "merlin-ldp-stg.cert",
        "daystoexpiration": 0,
        "key": "merlin-ldp-stg.abc.com.key"
    },
    {
        "cert": "stg-abc.services.cer",
        "certkey": "stg-abc.services.cert",
        "daystoexpiration": 0,
        "key": "stg-abc.services.key"
    },
    {
        "cert": "fintech-ap-stg.cer",
        "certkey": "fintech-ap-stg-cer",
        "daystoexpiration": 0,
        "key": "fintech-ap-stg.key"
    },
    {
        "cert": "docker.prod.abc.com_2021.cer",
        "certkey": "docker.prod.abc.com",
        "daystoexpiration": 0,
        "key": "docker.prod.abc.com_2021.key"
    },
    {
        "cert": "merlin-ldp.cert-2023",
        "certkey": "merlin-ldp.cert-2023",
        "daystoexpiration": 0,
        "key": "merlin-ldp.cert-2023"
    },
    {
        "cert": "abc-net-etc-2023",
        "certkey": "abc-net-etc-2023",
        "daystoexpiration": 0,
        "key": "abc-net-etc-2023"
    },
    {
        "cert": "ppc01.abc.com_2024.cer",
        "certkey": "ppc01.abc-2024",
        "daystoexpiration": 0,
        "key": "ppc01.abc_2024.key"
    }
]

2. certkey_binding列表（存储certkey与服务器绑定关系）：

[
    {
        "certkey": "help.abc.com-key",
        "data": "1",
        "servername": "dfw-xyz.help.abc.com-SSL_tcp443-lb",
        "stateflag": "2",
        "version": 2
    },
    {
        "certkey": "prod.abc.ca",
        "data": "1",
        "servername": "cca-canada.dfw.prod.abc.com-SSL_443tcp-lb",
        "stateflag": "2",
        "version": 2
    },
    {
        "certkey": "ca-profile-service-prod-cert",
        "data": "1",
        "servername": "dfw-ca-profile-service-prod.abc.com_ssl_443_lb",
        "stateflag": "2",
        "version": 2
    },
    {
        "certkey": "eclosing-cert",
        "data": "1",
        "servername": "eclosing.dfw.abc.com-SSL_443tcp-lb",
        "stateflag": "2",
        "version": 2
    },
    {
        "certkey": "merlin-ldp-stg.cert",
        "data": "1",
        "servername": "dfw.merlin-ldp-stg.abc.com-SSL_443tcp-lb",
        "stateflag": "2",
        "version": 2
    },
    {
        "certkey": "stg-abc.services.cert",
        "data": "1",
        "servername": "stg-abc.services.abc.com-DFW-SSL_443tcp-lb",
        "stateflag": "2",
        "version": 2
    },
    {
        "certkey": "fintech-ap-stg-cer",
        "data": "1",
        "servername": "dfw-fintech-ap-stg.abc.com-HTTPS_443tcp-lb",
        "stateflag": "2",
        "version": 2
    },
    {
        "certkey": "docker.prod.abc.com",
        "data": "1",
        "servername": "dfw-thirdparty.docker.prod.abc.com-SSL_443tcp-lb",
        "stateflag": "2",
        "version": 2
    },
    {
        "certkey": "docker.prod.abc.com",
        "data": "2",
        "servername": "dfw-us.gcr.docker.abc.com-SSL_tcp443-lb",
        "stateflag": "2",
        "version": 2
    },
    {
        "certkey": "docker.prod.abc.com",
        "data": "3",
        "servername": "dfw-elasticsearch.prod.abc.com-SSL_443tcp-lb",
        "stateflag": "2",
        "version": 2
    },
    {
        "certkey": "merlin-ldp.cert-2023",
        "data": "1",
        "servername": "merlin-ldp.merlin-ldp-DFW-SSL_443tcp-lb",
        "stateflag": "2",
        "version": 2
    },
    {
        "certkey": "abc-net-etc-2023",
        "data": "1",
        "servername": "abc-net-etc.dfw.prod.abc.com-SSL_443tcp-lb",
        "stateflag": "2",
        "version": 2
    },
    {
        "certkey": "ppc01.abc-2024",
        "data": "1",
        "servername": "dfw-clho.abc.com-SSL_443tcp-lb",
        "stateflag": "2",
        "version": 2
    }
]

需求 #

从expired列表中剔除所有在certkey_binding列表中存在服务器绑定的certkey，由于所有certkey均有绑定，预期filtered_certs为空列表。

当前问题 #

使用以下Ansible playbook后，filtered_certs仍包含expired列表的所有certkey，未达到预期：

- hosts: citrix_adc
  gather_facts: False

  tasks:
    - name: Filter expired certs with no binding
      set_fact:
        filtered_certs: "{{ expired | rejectattr('certkey','equalto',item.certkey) | list }}"
      loop: "{{ certkey_binding }}"
      
    - name: Write filtered certs into file
      copy:
        content: "{{ filtered_certs | to_nice_json }}"
        dest: '/Users/abcdef/Downloads/Ansible_Automation/NS_Cert_Cleanup/Certkeys_To_Delete.json'

问题原因 #

1. 循环中每次执行set_fact时，都是基于原始的expired列表进行过滤，而非在上一次过滤结果的基础上继续处理。这意味着每次循环只会剔除当前item.certkey对应的条目，之前循环剔除的条目会被重新包含进来，最终仅剩下最后一次循环剔除的条目，其他条目全部恢复。
2. certkey_binding中存在重复的certkey（如docker.prod.abc.com出现3次），循环处理重复值会做无效操作，且进一步干扰最终结果。

修正方案 #

先提取certkey_binding中所有唯一的certkey，再一次性过滤expired列表中属于该集合的条目。

修正后的Playbook #

- hosts: citrix_adc
  gather_facts: False

  tasks:
    - name: Extract all unique bound certkeys
      set_fact:
        bound_certkeys: "{{ certkey_binding | map(attribute='certkey') | unique | list }}"

    - name: Filter expired certs with no binding
      set_fact:
        filtered_certs: "{{ expired | rejectattr('certkey', 'in', bound_certkeys) | list }}"
      
    - name: Write filtered certs into file
      copy:
        content: "{{ filtered_certs | to_nice_json }}"
        dest: '/Users/abcdef/Downloads/Ansible_Automation/NS_Cert_Cleanup/Certkeys_To_Delete.json'

解释 #

1. 提取唯一绑定certkey：
   • map(attribute='certkey')：从certkey_binding列表的每个对象中提取certkey字段，得到一个包含所有certkey的列表。
   • unique：对上述列表去重，避免重复处理同一个certkey。
2. 一次性过滤：
   • rejectattr('certkey', 'in', bound_certkeys)：直接剔除expired列表中certkey存在于bound_certkeys集合的条目，最终得到没有绑定关系的过期证书列表。由于所有certkey均有绑定，filtered_certs会是空列表，符合预期。

内容的提问来源于stack exchange，提问作者abazaz