基于HTTP API构建Grafana仪表盘及Telegraf数据采集配置问题

需求概述 #

• 原使用InfluxDB作为Grafana数据源，现需从HTTP API（示例地址：https://example:8080/cluster）获取JSON数据，或通过Telegraf提取后发送到InfluxDB，目标是在Grafana中展示组件状态（如TEJD状态UP、TECJ状态DOWN）

目标API返回的JSON示例 #

{ "status": "UP",  "TEJD": {"status":"UP", "count":"0", "minDateTime":"", "description":"Workbasket Jdmin"},  "TECJ":
     {"status":"DOWN", "count":"0", "minDateTime":"",
     "description":"Workbasket CreateJppWait"},  "TEDE": {"status":"UP",
     "count":"0", "minDateTime":"", "description":"Workbasket default"}, 
     "TEEW": {"status":"UP", "count":"0", "minDateTime":"",
     "description":"Workbasket eFormWriteFailure"},  "TEFB":
     {"status":"UP", "count":"0", "minDateTime":"",
     "description":"Workbasket BackgroundProcessing"},  "TEIC":
     {"status":"UP", "count":"0", "minDateTime":"",
     "description":"Workbasket IncompleteConnections"},  "TELB":
     {"status":"UP", "count":"18", "minDateTime":"20/12/2022 10:37",
     "description":"Workbasket LRBackgroundProcess"},   "JETE":
     {"status":"UP", "count":"0", "minDateTime":"",
     "description":"Jssignment errors for Jssign-WorkBasket"},  "JEWL":
     {"status":"DOWN", "count":"0", "minDateTime":"",
     "description":"Jssignment errors for Jssign-Worklist"},   "FETE":
     {"status":"UP", "count":"0", "minDateTime":"", "description":"Flow
     errors for Jssign-WorkBasket"},  "FEWL": {"status":"UP", "count":"0",
     "minDateTime":"", "description":"Flow errors for Jssign-Worklist"},  
     "BQBP": {"status":"UP", "count":"0", "minDateTime":"",
     "description":"Broken queue System-Queue-BackgroundProcess"},  "BQDE":
     {"status":"UP", "count":"0", "minDateTime":"", "description":"Broken
     queue System-Queue-DefaultEntry"},   "CQCJ": {"status":"UP",
     "count":"0", "minDateTime":"", "description":"Custom query Create Jpp
     Requests"},   "JS02": {"status":"UP", "description":"Job Scheduler
     CaseDocumentDeletion (Jny one associated node:BackgroundProcessing)"},
     "JS21": {"status":"UP", "description":"Job Scheduler
     PurgeOldSIDExchangeRecords (Jll associated
     nodes:BackgroundProcessing)"},  "JS01": {"status":"UP",
     "description":"Job Scheduler UpdateReferenceData (Jny one associated
     node:BackgroundProcessing)"},   "package": { "name": "Pega Cluster",
     "elapsedTime": "819.0", "lastUpdated": "20/12/2022 10:52" } }

当前尝试与问题 #

尝试使用Telegraf的HTTP插件解析数据，但配置后无法正确提取组件标识（如TEJD、TECJ），导致Grafana无法关联对应组件状态。

错误的Telegraf配置 #

[[inputs.http]]
    urls = ["http://example:8080//Cluster"]
    tagexclude = ["url", "host"]
    #Overwrite measurement name from default `http` to `cluster`
    ### name_override = "httppegacluster"
    data_format = "json_v2"
    [[inputs.http.json_v2]]
        [[inputs.http.json_v2.object]]
             path = "test1"
           

        [[inputs.http.json_v2.object]]
             path = "test2"
        
        [[inputs.http.json_v2.object]]
            path = "test3"
           
        [[inputs.http.json_v2.object]]
            path = "test4"
          
        [[inputs.http.json_v2.object]]
            path = "test5"
    

测试输出结果 #

执行命令 telegraf -config http.conf -test 后，输出未包含组件标识：

> http count="0",description="Workbasket Admin",minDateTime="",status="UP" 1671286903000000000
> http count="0",description="Workbasket eFormWriteFailure",minDateTime="",status="UP" 1671286903000000000
> http count="0",description="Workbasket CreateAppWait",minDateTime="",status="UP" 1671286903000000000
> http count="0",description="Workbasket default@RoS",minDateTime="",status="UP" 1671286903000000000
> http count="0",description="Workbasket BackgroundProcessing",minDateTime="",status="UP" 1671286903000000000

解决方案 #

问题出在JSON解析配置上，原配置的path设置错误，没有正确匹配JSON中的组件键（如TEJD、TECJ）。需要使用json_v2的遍历规则，将组件名作为标签提取。

正确的Telegraf配置示例 #

[[inputs.http]]
    urls = ["http://example:8080/Cluster"]  # 修正URL的双斜杠问题
    tagexclude = ["url", "host"]
    name_override = "cluster_status"  # 自定义测量名称，方便Grafana识别
    data_format = "json_v2"

    [[inputs.http.json_v2]]
        # 遍历顶层所有组件对象，排除根节点的status和package
        [[inputs.http.json_v2.object]]
            path = "@this"
            exclude_keys = ["status", "package"]
            # 将每个组件的键名（如TEJD）作为tag
            tag_keys = ["component"]
            # 提取组件的属性字段
            field_keys = ["status", "count", "minDateTime", "description"]

        # 可选：提取根节点的整体集群状态
        [[inputs.http.json_v2.object]]
            path = "status"
            field_name = "overall_status"

        # 可选：提取package信息
        [[inputs.http.json_v2.object]]
            path = "package"
            field_keys = ["name", "elapsedTime", "lastUpdated"]

配置说明 #

1. 修正URL路径：去掉多余的//，避免请求路径错误
2. 自定义测量名：用name_override将默认的http改为cluster_status，更直观区分数据源
3. 遍历顶层组件：path = "@this"会遍历根节点下所有键，排除不需要的status和package
4. 组件名作为标签：tag_keys = ["component"]自动将每个组件的键（如TEJD）转为标签，方便Grafana筛选不同组件
5. 提取核心字段：field_keys指定需要采集的组件属性，确保Grafana能获取状态、计数等信息

测试验证 #

执行telegraf -config http.conf -test后，输出会包含component标签，示例如下：

> cluster_status,component=TEJD count="0",description="Workbasket Jdmin",minDateTime="",status="UP" 1671286903000000000
> cluster_status,component=TECJ count="0",description="Workbasket CreateJppWait",minDateTime="",status="DOWN" 1671286903000000000

Grafana展示配置 #

在Grafana中：

• 选择InfluxDB数据源，测量名选择cluster_status
• 使用component标签筛选目标组件
• 用状态类面板（如Single Stat、Status Panel）展示status字段，设置UP为绿色、DOWN为红色等可视化规则

内容的提问来源于stack exchange，提问作者user5820327