如何在Ansible中按主机提取关联服务的日志文件数据？

现有清单 #

---
all:
  children:
vms:
  hosts:
    vm1:
      services:
      - exim
      - apache
    vm2:
      services:
      - exim
      - nginx
    vm3:
      services:
      - exim
      - nginx

定义的变量 #

services_parameters:
  "nginx":
    "log_files":
      - /var/log/nginx/*.log
    "fail2ban_rules":
      "nginx-limit-req":
      "nginx-http-auth":
      "nginx-botsearch":
      "nginx-bad-request":
  "exim":
    "log_files":
      - /var/log/exim4/mainlog
    "fail2ban_rules":
      "exim":
      "exim-spam":
  "apache":
    "log_files":
      - /var/log/apache2/*.log
    "fail2ban_rules":
      "apache-botsearch":
      "apache-auth":
        "parameters":
          "logpath": "/var/log/apache2/*.log"
          "findtime": 3600
          "bantime": 86400
          "maxretry": 3
      "apache-badbots":

已实现的操作 #

通过以下表达式成功提取并合并了各主机的fail2ban规则：

"{{ hostvars[inventory_hostname].services | map('extract', services_parameters, 'fail2ban_rules') | combine }}"

得到的事实结果如下：

ok: [vm1] => {
    "fail2ban_rules": {
        "apache-auth": {
            "parameters": {
                "logpath": "/var/log/apache2/*.log"
            }
        },
        "apache-badbots": null,
        "apache-botsearch": null,
        "exim": null,
        "exim-spam": null
    }
}
ok: [vm2] => {
    "fail2ban_rules": {
        "exim": null,
        "exim-spam": null,
        "nginx-bad-request": null,
        "nginx-botsearch": null,
        "nginx-http-auth": null,
        "nginx-limit-req": null
    }
}
ok: [vm3] => {
    "fail2ban_rules": {
        "exim": null,
        "exim-spam": null,
        "nginx-bad-request": null,
        "nginx-botsearch": null,
        "nginx-http-auth": null,
        "nginx-limit-req": null
    }
}

需求 #

现在需要提取每个主机对应的所有log_files并关联到对应服务，期望得到如下格式的事实：

ok: [vm1] => [
      {"exim": [
        "/var/log/exim4/mainlog"
        ]
      },
      {"apache": [
        "/var/log/apache2/*.log"
        ]
      }
]
ok: [vm2] => [
      {"exim": [
        "/var/log/exim4/mainlog"
        ]
      },
      {"nginx": [
        "/var/log/nginx/*.log"
        ]
      }
]
ok: [vm3] => [
      {"exim": [
        "/var/log/exim4/mainlog"
        ]
      },
      {"nginx": [
        "/var/log/nginx/*.log"
        ]
      }
]

后续计划基于该事实循环生成如下配置块：

nginx:
  paths:
    - /var/log/nginx/*.log
    - <other paths if defined in services_parameters variable>

解决方案 #

可以通过Jinja2的列表推导结合set_fact模块实现需求：

提取服务日志路径的Playbook代码 #

- name: 提取主机对应服务的日志文件路径
  set_fact:
    service_logs: >-
      {% set log_list = [] %}
      {% for service in services %}
        {% do log_list.append({service: services_parameters[service].log_files}) %}
      {% endfor %}
      {{ log_list }}

验证结果 #

执行后，service_logs事实会生成你期望的列表格式，例如vm1的结果为：

service_logs:
  - exim:
      - /var/log/exim4/mainlog
  - apache:
      - /var/log/apache2/*.log

生成配置块 #

使用Ansible模板（Template）遍历service_logs即可生成目标配置：

{% for item in service_logs %}
{% for svc, paths in item.items() %}
{{ svc }}:
  paths:
    {% for path in paths %}
    - {{ path }}
    {% endfor %}
{% endfor %}
{% endfor %}

该模板会输出如下格式的内容（以vm2为例）：

exim:
  paths:
    - /var/log/exim4/mainlog
nginx:
  paths:
    - /var/log/nginx/*.log

内容的提问来源于stack exchange，提问作者jeanlabab