promtool单元测试失败:MySQL告警仅评估输入序列首个值
问题:Prometheus单元测试中MySQLReplicationDown告警未触发
我在进行Prometheus单元测试时,多次调整input_series后仍持续失败,测试似乎仅评估输入序列中的首个值。具体情况如下:
单元测试文件
tests/infra_db_alerts-tests.yml
失败信息
FAILED:
alertname: MySQLReplicationDown, time: 10m,
exp:[
0:
Labels:{alert_owner="alerts-infra", alertname="MySQLReplicationDown", severity="High"}
Annotations:{description="MySQL replication is broken. \n"}
],
got:[]
我的Prometheus测试规则
rule_files: - ../rules/db_alerts.yml tests: # MySQL Alert Tests - input_series: - series: mysql_up values: 0 - series: mysql_global_variables_innodb_replication_delay values: '1 1 1 1 1 0 0 0 0 -1 -1 -1 -1 -1 -1' alert_rule_test: - alertname: MySQLReplicationLag eval_time: 5m exp_alerts: - exp_labels: severity: High alert_owner: alerts-infra exp_annotations: description: "There is a MySQL replication delay. \n" - alertname: MySQLReplicationDown eval_time: 10m exp_alerts: - exp_labels: severity: High alert_owner: alerts-infra exp_annotations: description: "MySQL replication is broken. \n"
问题根源分析
mysql_up指标时间覆盖不足:仅给mysql_up设置单个值0,意味着该指标仅在测试起始时间(0m)有样本,后续时间点(如10m)无持续数据。若MySQLReplicationDown规则依赖mysql_up == 0持续一段时间,后续无数据会导致规则判定无有效样本,无法触发告警。- 输入序列时间步长不匹配:Prometheus单元测试中,
values的每个数值默认对应1m时间步长。你给mysql_global_variables_innodb_replication_delay设置了15个值(覆盖到14m),但mysql_up仅覆盖0m,两者时间范围不匹配。
修复方案
- 补全
mysql_up序列值:将mysql_up的values设置为与mysql_global_variables_innodb_replication_delay相同长度的0序列,确保10m时该指标仍有有效样本:
- series: mysql_up values: '0 0 0 0 0 0 0 0 0 0 0 0 0 0 0'
- 核对告警规则逻辑:检查
../rules/db_alerts.yml中MySQLReplicationDown的规则,确认是否要求mysql_up持续为0达到指定时长,或结合mysql_global_variables_innodb_replication_delay = -1的条件,确保测试输入序列在10m时满足所有触发条件。
内容的提问来源于stack exchange,提问作者Rad4
相关产品推荐
相关产品推荐

