Windows_exporter服务监控：静态配置允许停止的服务查询改造

问题背景 #

原使用windows_exporter的PromQL查询语句如下，原本想通过动态方式标记允许停止的服务，但手动终止服务也会触发stopped状态，导致误报，因此需要改为静态硬编码，明确指定哪些服务器上的服务处于停止状态是正常的：

(windows_service_state{app="xxx", name=~"tomcat.+|ot.+", state="running"} == 0) - ignoring(state) (windows_service_state{app="xxx", name=~"tomcat.+|ot.+", state="stopped"} == 0)

改造实现方法 #

将查询的第二部分替换为静态匹配允许停止的服务实例，通过精确匹配instance和name标签，只排除那些确认应该处于停止状态的服务，避免误报。改造后的完整查询语句如下：

(windows_service_state{app="xxx", name=~"tomcat.+|ot.+", state="running"} == 0) 
- ignoring(state) 
(
  windows_service_state{app="xxx", instance="otcs-olap-adm1.internal-08.example.org", name="otcs", state="stopped"} == 0
  OR windows_service_state{app="xxx", instance="otcs-olap-adm1.internal-08.example.org", name="otcsadmin", state="stopped"} == 0
  OR windows_service_state{app="xxx", instance="otcs-int-f1.internal-08.example.org", name="otcs", state="stopped"} == 0
  OR windows_service_state{app="xxx", instance="otcs-int-f2.internal-08.example.org", name="otcs", state="stopped"} == 0
  OR windows_service_state{app="xxx", instance="otcs-olap-f1.internal-08.example.org", name="otcs", state="stopped"} == 0
  OR windows_service_state{app="xxx", instance="otcs-olap-f2.internal-08.example.org", name="otcs", state="stopped"} == 0
  OR windows_service_state{app="xxx", instance="otcs-olap-f3.internal-08.example.org", name="otcs", state="stopped"} == 0
  OR windows_service_state{app="xxx", instance="otcs-ooriom-f1.internal-21.example.org", name="otsystemcenteragent", state="stopped"} == 0
  OR windows_service_state{app="xxx", instance="otcs-olap-idx1.internal-08.example.org", name="otcs", state="stopped"} == 0
  OR windows_service_state{app="xxx", instance="otcs-olap-idx1.internal-08.example.org", name="otcsadmin", state="stopped"} == 0
  OR windows_service_state{app="xxx", instance="otcs-int-idxad1.internal-08.example.org", name="otcs", state="stopped"} == 0
  OR windows_service_state{app="xxx", instance="otcs-int-idxad1.internal-08.example.org", name="otcsadmin", state="stopped"} == 0
  OR windows_service_state{app="xxx", instance="otcs-ooriom-b1.internal-21.example.org", name="otcs", state="stopped"} == 0
  OR windows_service_state{app="xxx", instance="otcs-ooriom-b1.internal-21.example.org", name="otcsadmin", state="stopped"} == 0
  OR windows_service_state{app="xxx", instance="otcs-ooriom-b1.internal-21.example.org", name="otsystemcenteragent", state="stopped"} == 0
  OR windows_service_state{app="xxx", instance="otcs-int-src.internal-08.example.org", name="otcs", state="stopped"} == 0
  OR windows_service_state{app="xxx", instance="otcs-int-src.internal-08.example.org", name="otcsadmin", state="stopped"} == 0
  OR windows_service_state{app="xxx", instance="otcs-olap-src1.internal-08.example.org", name="otcs", state="stopped"} == 0
  OR windows_service_state{app="xxx", instance="otcs-olap-src1.internal-08.example.org", name="otcsadmin", state="stopped"} == 0
  OR windows_service_state{app="xxx", instance="otcs-olap-src2.internal-08.example.org", name="otcs", state="stopped"} == 0
  OR windows_service_state{app="xxx", instance="otcs-olap-src2.internal-08.example.org", name="otcsadmin", state="stopped"} == 0
  OR windows_service_state{app="xxx", instance="otds-int-f1.internal-08.example.org", name="tomcat10", state="stopped"} == 0
  OR windows_service_state{app="xxx", instance="otds-olap-f1.internal-08.example.org", name="tomcat10", state="stopped"} == 0
  OR windows_service_state{app="xxx", instance="otds-olap-f2.internal-08.example.org", name="tomcat10", state="stopped"} == 0
  OR windows_service_state{app="xxx", instance="otds-ooriom-f1.internal-21.example.org", name="tomcat10", state="stopped"} == 0
  OR windows_service_state{app="xxx", instance="otsc-ooriom-f1.internal-21.example.org", name="otsystemcentermanager", state="stopped"} == 0
)

允许停止的服务与服务器列表 #

按环境分类整理的正常停止服务清单：

生产环境（Prod） #

• otcs-olap-adm1.internal-08.example.org：otcs、otcsadmin
• otcs-olap-f1.internal-08.example.org：otcs
• otcs-olap-f2.internal-08.example.org：otcs
• otcs-olap-f3.internal-08.example.org：otcs
• otcs-olap-idx1.internal-08.example.org：otcs、otcsadmin
• otcs-olap-src1.internal-08.example.org：otcs、otcsadmin
• otcs-olap-src2.internal-08.example.org：otcs、otcsadmin
• otds-olap-f1.internal-08.example.org：tomcat10
• otds-olap-f2.internal-08.example.org：tomcat10

集成环境（Int） #

• otcs-int-f1.internal-08.example.org：otcs
• otcs-int-f2.internal-08.example.org：otcs
• otcs-int-idxad1.internal-08.example.org：otcs、otcsadmin
• otcs-int-src.internal-08.example.org：otcs、otcsadmin
• otds-int-f1.internal-08.example.org：tomcat10

测试环境（Test） #

• otcs-ooriom-f1.internal-21.example.org：otsystemcenteragent
• otcs-ooriom-b1.internal-21.example.org：otcs、otcsadmin、otsystemcenteragent
• otds-ooriom-f1.internal-21.example.org：tomcat10
• otsc-ooriom-f1.internal-21.example.org：otsystemcentermanager

内容的提问来源于stack exchange，提问作者Sybil