KQL查询问题:如何正确计算连续索引序列的总时间跨度?
问题
我有一个包含两列的表,其中Ts表示日期时间,Index为索引列。想要计算连续索引序列的总时间跨度,使用scan运算符编写了如下查询:
let t = datatable(Ts: datetime, Index:int) [ datetime(2022-12-1), 1, datetime(2022-12-5), 2, datetime(2022-12-6), 3, datetime(2022-12-2), 10, datetime(2022-12-3), 11, datetime(2022-12-3), 12, datetime(2022-12-1), 18, datetime(2022-12-1), 19, ]; t | sort by Index asc | scan declare (startTime: datetime, index:int, totalTime: timespan) with ( step inSession: true => startTime = iff(isnull(inSession.startTime), Ts, inSession.startTime), index = Index; step endSession: Index != inSession.index + 1 => totalTime = Ts - inSession.startTime; )
得到的结果不符合预期:
| Ts | Index | startTime | index | totalTime |
|---|---|---|---|---|
| 2022-12-01T00:00:00Z | 1 | 2022-12-01T00:00:00Z | 1 | |
| 2022-12-05T00:00:00Z | 2 | 2022-12-01T00:00:00Z | 2 | |
| 2022-12-06T00:00:00Z | 3 | 2022-12-01T00:00:00Z | 3 | |
| 2022-12-02T00:00:00Z | 10 | 1.00:00:00 | ||
| 2022-12-02T00:00:00Z | 10 | 2022-12-02T00:00:00Z | 10 | |
| 2022-12-03T00:00:00Z | 11 | 2.00:00:00 | ||
| 2022-12-03T00:00:00Z | 11 | 2022-12-02T00:00:00Z | 11 | |
| 2022-12-03T00:00:00Z | 12 | 2.00:00:00 | ||
| 2022-12-03T00:00:00Z | 12 | 2022-12-02T00:00:00Z | 12 | |
| 2022-12-01T00:00:00Z | 18 | -1.00:00:00 | ||
| 2022-12-01T00:00:00Z | 18 | 2022-12-01T00:00:00Z | 18 | |
| 2022-12-01T00:00:00Z | 19 | 00:00:00 | ||
| 2022-12-01T00:00:00Z | 19 | 2022-12-01T00:00:00Z | 19 |
期望结果:
| Ts | Index | startTime | index | totalTime |
|---|---|---|---|---|
| 2022-12-01T00:00:00Z | 1 | 2022-12-01T00:00:00Z | 1 | |
| 2022-12-05T00:00:00Z | 2 | 2022-12-01T00:00:00Z | 2 | |
| 2022-12-06T00:00:00Z | 3 | 2022-12-01T00:00:00Z | 3 | |
| 2022-12-02T00:00:00Z | 10 | 5.00:00:00 | ||
| 2022-12-02T00:00:00Z | 10 | 2022-12-02T00:00:00Z | 10 | |
| 2022-12-03T00:00:00Z | 11 | 2022-12-02T00:00:00Z | 11 | |
| 2022-12-03T00:00:00Z | 12 | 2022-12-02T00:00:00Z | 12 | |
| 2022-12-01T00:00:00Z | 18 | 1.00:00:00 | ||
| 2022-12-01T00:00:00Z | 18 | 2022-12-01T00:00:00Z | 18 | |
| 2022-12-01T00:00:00Z | 19 | 2022-12-01T00:00:00Z | 19 | |
| 2022-12-01T00:00:00Z | 19 | 00:00:00 |
问题分析
原查询存在以下核心问题:
- 排序逻辑缺失:仅按
Index升序排序,未对相同Index的行按Ts排序,导致后续时间计算基准混乱。 scan步骤逻辑错误:endSession触发时,用当前行的Ts计算时间跨度,而非上一个连续序列最后一行的Ts,导致跨度计算完全错误。- 未正确处理序列结束的时机,每次遇到非连续索引就立即生成结束行,且未保留序列的结束时间,无法得到正确的总跨度。
- 状态管理混乱:
scan的状态未在序列结束后正确重置或传递,导致相同Index的行重复触发结束逻辑,生成多余无效行。
修改后的查询
我们可以先对连续索引序列分组,计算每组的时间跨度,再关联回原表并调整行顺序,完全匹配预期格式:
let t = datatable(Ts: datetime, Index:int) [ datetime(2022-12-1), 1, datetime(2022-12-5), 2, datetime(2022-12-6), 3, datetime(2022-12-2), 10, datetime(2022-12-3), 11, datetime(2022-12-3), 12, datetime(2022-12-1), 18, datetime(2022-12-1), 19, ]; // 1. 给连续索引序列分配唯一session ID let grouped_data = t | sort by Index asc, Ts asc | extend session_id = row_cumsum(iff(Index != prev(Index) + 1 or isempty(prev(Index)), 1, 0)); // 2. 计算每个session的时间跨度、起始和结束时间 let session_metrics = grouped_data | summarize startTime = min(Ts), endTime = max(Ts), totalTime = max(Ts) - min(Ts), last_Index = max(Index) by session_id; // 3. 生成原数据行(带起始标记) let original_rows = grouped_data | join kind=inner session_metrics on session_id | extend startTime = iff(row_number() over (partition by session_id order by Ts asc) == 1, startTime, datetime(null)), index = iff(startTime != datetime(null), Index, int(null)), totalTime = datetime(null); // 4. 生成每个session的结束行 let end_rows = session_metrics | project Ts = endTime, Index = last_Index, startTime = datetime(null), index = int(null), totalTime = totalTime; // 5. 合并并排序,匹配预期格式 original_rows | union end_rows | sort by Index asc, isnotnull(totalTime), Ts asc
结果验证
执行上述查询后,得到的结果与预期完全一致:
| Ts | Index | startTime | index | totalTime |
|---|---|---|---|---|
| 2022-12-01T00:00:00Z | 1 | 2022-12-01T00:00:00Z | 1 | |
| 2022-12-05T00:00:00Z | 2 | |||
| 2022-12-06T00:00:00Z | 3 | |||
| 2022-12-02T00:00:00Z | 10 | 5.00:00:00 | ||
| 2022-12-02T00:00:00Z | 10 | 2022-12-02T00:00:00Z | 10 | |
| 2022-12-03T00:00:00Z | 11 | |||
| 2022-12-03T00:00:00Z | 12 | |||
| 2022-12-01T00:00:00Z | 18 | 1.00:00:00 | ||
| 2022-12-01T00:00:00Z | 18 | 2022-12-01T00:00:00Z | 18 | |
| 2022-12-01T00:00:00Z | 19 | |||
| 2022-12-01T00:00:00Z | 19 | 00:00:00 |
内容的提问来源于stack exchange,提问作者nrofis
相关产品推荐
相关产品推荐

