如何在Weights & Biases自定义图表中用Vega-Lite实现归一化频率直方图

目标 #

在Weights & Biases（基于Vega-Lite）的自定义图表中，将分组直方图归一化，使每组柱高之和为1（因箱宽设为1，结果同时为有效PDF/PMF）。目前已实现非归一化版本，但添加归一化逻辑后图表无法显示。

当前尝试 #

参考Vega-Lite文档，尝试在transform块末尾添加joinaggregate求和总计数，再通过calculate计算相对频率，但添加后图表失效。

问题代码片段 #

添加的归一化逻辑代码：

{
  "joinaggregate": [
    {"op": "sum", "field": "Count", "as": "TotalCount"}
  ],
  "groupby": ["newGroupKeys", "color", "grouped"]
},
{
  "calculate": "datum.Count / datum.TotalCount",
  "as": "RelativeFrequency"
}

可用的非归一化Vega-Lite代码 #

{
  "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
  "description": "A simple histogram",
  "data": {
    "name": "wandb"
  },
   "transform": [
    {
      "calculate": "if('${field:groupKeys}' === ''  || datum['${field:groupKeys}'] === '', false, true)",
      "as": "grouped"
    },
    {
      "calculate": "if('${field:groupKeys}' === ''  || datum['${field:groupKeys}'] === '', datum.name, datum['${field:groupKeys}'])",
      "as": "newGroupKeys"
    },
    {
      "calculate": "if('${field:groupKeys}' === ''  || datum['${field:groupKeys}'] === '', datum.color, datum['${field:groupKeys}'])",
      "as": "color"
    },
  {
    "aggregate": [
      {
      "op" : "average",
      "field": "${field:value}",
      "as": "${field:value}"
      }
    ],
    "groupby": ["newGroupKeys", "color", "grouped", "${field:value}"]
  }
],
  "selection": {
    "grid": {
      "type": "interval", "bind": "scales"
    }
  },
  "title": "${string:title}",
  "layer": [
    {
      "transform": [
        {"filter": "datum.grouped == false"}
      ],
      "mark": {"type": "bar", "tooltip": {"content": "data"}},
      "encoding": {
        "x": {
          "bin": {"binned" : false, "step" : 1},
          "type": "quantitative",
          "field": "${field:value}"
        },
        "y": {
          "aggregate": "count",
          "stack": null
        },
        "opacity": {"value": 0.6},
        "detail": [{"field": "newGroupKeys"}, {"field": "color"}],
        "color": {
          "type": "nominal",
          "field": "newGroupKeys",
          "scale": {"range": {"field": "color"}},
          "legend": {"title": null}
        }
      }
    },
    {
      "transform": [
        {"filter": "datum.grouped == true"}
      ],
      "mark": {"type": "bar", "binSpacing": 0, "tooltip": {"content": "data"}, "clip": true},
      "encoding": {
        "x": {
          "bin" : {"binned" : false, "step" : 1}, 
          "type": "quantitative",
          "scale": {"domain": [0, 30]},
          "field": "${field:value}"
        },
        "y": {
          "aggregate": "count",
          "stack": null
        },
        "opacity": {"value": 0.6},
        "detail": [{"field": "newGroupKeys"}, {"field": "color"}],
        "color": {
          "field": "newGroupKeys",
          "type": "nominal",
          "scale": {"range": "category"},
          "legend": {"title": null}
        }
      }
    }
  ],
  "resolve": {"scale": {"color": "independent"}}
}

数据示例 #

{"data": { "values": [
    {"uturns/uturns": 0.3, "groupKeys": "group1", "grouped": true},
    {"uturns/uturns": 2.8, "groupKeys": "group1", "grouped": true},
    {"uturns/uturns": 1.7, "groupKeys": "group2", "grouped": true},
    {"uturns/uturns": 0.8, "groupKeys": "group2", "grouped": true},
]}}

解决方法 #

问题核心在于现有聚合逻辑与归一化步骤不匹配，且编码层仍在使用aggregate: count，导致数据处理流程冲突。按以下步骤修改：

修改后的完整代码 #

{
  "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
  "description": "Normalized frequency histogram",
  "data": {
    "name": "wandb"
  },
  "transform": [
    // 保留原有分组字段计算逻辑
    {
      "calculate": "if('${field:groupKeys}' === ''  || datum['${field:groupKeys}'] === '', false, true)",
      "as": "grouped"
    },
    {
      "calculate": "if('${field:groupKeys}' === ''  || datum['${field:groupKeys}'] === '', datum.name, datum['${field:groupKeys}'])",
      "as": "newGroupKeys"
    },
    {
      "calculate": "if('${field:groupKeys}' === ''  || datum['${field:groupKeys}'] === '', datum.color, datum['${field:groupKeys}'])",
      "as": "color"
    },
    // 第一步：对目标字段分箱（step=1与需求一致）
    {
      "bin": {"step": 1},
      "field": "${field:value}",
      "as": "binned_value"
    },
    // 第二步：按分组+分箱结果聚合计数
    {
      "aggregate": [{"op": "count", "as": "Count"}],
      "groupby": ["newGroupKeys", "color", "grouped", "binned_value"]
    },
    // 第三步：按分组计算每组总计数
    {
      "joinaggregate": [{"op": "sum", "field": "Count", "as": "TotalCount"}],
      "groupby": ["newGroupKeys", "color", "grouped"]
    },
    // 第四步：计算归一化频率
    {
      "calculate": "datum.Count / datum.TotalCount",
      "as": "RelativeFrequency"
    }
  ],
  "selection": {
    "grid": {
      "type": "interval", "bind": "scales"
    }
  },
  "title": "${string:title}",
  "layer": [
    {
      "transform": [{"filter": "datum.grouped == false"}],
      "mark": {"type": "bar", "tooltip": {"content": "data"}},
      "encoding": {
        "x": {
          "type": "quantitative",
          "field": "binned_value",
          "title": "${field:value}"
        },
        "y": {
          "type": "quantitative",
          "field": "RelativeFrequency",
          "stack": null,
          "title": "Normalized Frequency"
        },
        "opacity": {"value": 0.6},
        "color": {
          "type": "nominal",
          "field": "newGroupKeys",
          "scale": {"range": {"field": "color"}},
          "legend": {"title": null}
        }
      }
    },
    {
      "transform": [{"filter": "datum.grouped == true"}],
      "mark": {"type": "bar", "binSpacing": 0, "tooltip": {"content": "data"}, "clip": true},
      "encoding": {
        "x": {
          "type": "quantitative",
          "field": "binned_value",
          "scale": {"domain": [0, 30]},
          "title": "${field:value}"
        },
        "y": {
          "type": "quantitative",
          "field": "RelativeFrequency",
          "stack": null,
          "title": "Normalized Frequency"
        },
        "opacity": {"value": 0.6},
        "color": {
          "field": "newGroupKeys",
          "type": "nominal",
          "scale": {"range": "category"},
          "legend": {"title": null}
        }
      }
    }
  ],
  "resolve": {"scale": {"color": "independent"}}
}

关键修改点说明 #

1. 提前分箱处理：先对目标字段执行bin转换，确保后续计数是基于分箱后的分组，符合直方图的逻辑
2. 调整聚合顺序：先统计每个分箱的计数，再通过joinaggregate计算每组的总计数，最后得到归一化值
3. 编码层去聚合：将原y: {aggregate: count}替换为直接映射计算好的RelativeFrequency字段，避免重复聚合导致的数据冲突

内容的提问来源于stack exchange，提问作者Gilad Turok