Prometheus Alertmanager配置调整:指定环境规则的告警邮箱修改
问题描述
现有Prometheus监控已针对10个环境、10条规则完成配置,所有告警邮件统一发送至aaa@aaa.jp。现需新增配置,让3个特定环境的3条规则将告警邮件发送至另一邮箱bbb@bbb.jp。以下是当前Alertmanager配置及之前尝试但未成功的配置,需指导正确的配置添加方法。
当前Alertmanager配置
global: smtp_smarthost: 'email-smtp.ap-northeast-1.amazonaws.com:111' smtp_from: 'Alertmanager <aaa@abcd.jp>' smtp_auth_username: 'ASDFGHJKLQWERTYUIO' smtp_auth_password: 'hfiytOUYiuhlIYFTDilkUTFRDikhgfG' # smtp_auth_secret: '*********' route: # 默认接收器 receiver: 'default' # 按告警名、实例名、严重度分组,判定同类告警的条件 group_by: ['alertname', 'instance', 'severity'] # 同类告警合并为一组的等待时间 group_wait: 30s # 同类告警再次通知的间隔时间(含恢复通知) group_interval: 5m # 已通知告警的重复通知间隔 repeat_interval: 1h routes: # 按告警严重度切换发送目标 - match: severity: critical group_by: ['alertname', 'instance', 'severity'] group_wait: 30s group_interval: 5m repeat_interval: 1h receiver: critical - match: severity: warning group_by: ['alertname', 'instance', 'severity'] group_wait: 30s group_interval: 5m repeat_interval: 1h receiver: warning receivers: - name: 'default' email_configs: - to: aaa@aaa.jp send_resolved: true - name: 'critical' email_configs: - to: aaa@aaa.jp send_resolved: true - name: 'warning' slack_configs: # Slack配置保留原有注释内容 inhibit_rules: - source_match: # 当告警严重度为critical时 severity: 'critical' target_match: # 同一告警名的warning告警不发送通知 severity: 'warning' equal: ['alertname']
之前尝试的错误配置
route: - match: job_name = www.abcdefg.com -alert_name = export_down group_by: ['alertname', 'instance', 'severity'] group_wait: 30s group_interval: 5m repeat_interval: 1h receiver: customer receivers: - name: 'customer' email_configs: - to: bbb@bbb.jp send_resolved: true
错误分析与正确配置方法
错误原因
之前的配置存在多处语法与规则错误:
- YAML语法错误:匹配条件使用
=而非YAML标准的:,缩进混乱且包含全角空格,导致配置无法解析。 - 标签名错误:Alertmanager中无
job_name和alert_name标签,正确标签应为job(对应Prometheus采集任务名)和alertname(对应告警规则名)。 - 路由结构错误:新增路由直接覆盖原有
route节点,而非插入到route.routes数组中,导致原有规则全部失效。
正确配置步骤
步骤1:新增专属接收器
在receivers节点末尾添加针对bbb@bbb.jp的接收器:
- name: 'customer' email_configs: - to: bbb@bbb.jp send_resolved: true
步骤2:添加高优先级路由规则
Alertmanager路由按从上到下顺序匹配,需将特定环境/规则的路由放在routes数组最前面,确保优先匹配。
假设3个特定环境的job标签值为env-1、env-2、env-3,3条规则的alertname为rule-a、rule-b、rule-c,添加如下路由:
routes: # 新增:匹配特定环境与规则的路由(优先级最高) - matchers: - job =~ "env-1|env-2|env-3" - alertname =~ "rule-a|rule-b|rule-c" group_by: ['alertname', 'instance', 'severity'] group_wait: 30s group_interval: 5m repeat_interval: 1h receiver: customer # 原有critical路由(保持不变) - match: severity: critical ...
说明:
- 使用
matchers支持正则匹配,可一次性匹配多个环境/规则;若需精确匹配单个值,可使用match(如match: {job: "env-1"})。- 保持分组、间隔参数与原有规则一致,确保告警逻辑统一。
完整修改后的配置
global: smtp_smarthost: 'email-smtp.ap-northeast-1.amazonaws.com:111' smtp_from: 'Alertmanager <aaa@abcd.jp>' smtp_auth_username: 'ASDFGHJKLQWERTYUIO' smtp_auth_password: 'hfiytOUYiuhlIYFTDilkUTFRDikhgfG' # smtp_auth_secret: '*********' route: receiver: 'default' group_by: ['alertname', 'instance', 'severity'] group_wait: 30s group_interval: 5m repeat_interval: 1h routes: # 新增:特定环境+特定规则的告警路由 - matchers: - job =~ "env-1|env-2|env-3" - alertname =~ "rule-a|rule-b|rule-c" group_by: ['alertname', 'instance', 'severity'] group_wait: 30s group_interval: 5m repeat_interval: 1h receiver: customer # 原有critical路由 - match: severity: critical group_by: ['alertname', 'instance', 'severity'] group_wait: 30s group_interval: 5m repeat_interval: 1h receiver: critical # 原有warning路由 - match: severity: warning group_by: ['alertname', 'instance', 'severity'] group_wait: 30s group_interval: 5m repeat_interval: 1h receiver: warning receivers: - name: 'default' email_configs: - to: aaa@aaa.jp send_resolved: true - name: 'critical' email_configs: - to: aaa@aaa.jp send_resolved: true - name: 'warning' slack_configs: # Slack配置保留原有注释内容 # 新增:专属接收器 - name: 'customer' email_configs: - to: bbb@bbb.jp send_resolved: true inhibit_rules: - source_match: severity: 'critical' target_match: severity: 'warning' equal: ['alertname']
验证配置
修改完成后,执行以下命令验证语法正确性:
amtool config validate --config-file=alertmanager.yml
若输出Config file is valid则配置正确,重启Alertmanager即可生效。
内容的提问来源于stack exchange,提问作者Leo BM
相关产品推荐
相关产品推荐

