如何在Kusto查询中实现字段向下填充(fill-down)功能?
实现Kusto查询的字段向下填充(Fill-Down)需求
当然可以实现!在Kusto(Azure Data Explorer)中,你可以借助scan运算符轻松完成这种向下填充最后一个非空值的需求——它专门设计用来处理需要逐行跟踪状态(比如这里的最近有效版本号)的序列场景。
修改后的完整查询
datatable (Date:datetime, LogEntry:string) [ datetime(1910-06-11), "version: 1.0", datetime(1930-01-01), "starting foo", datetime(1953-01-01), "ending foo", datetime(1910-06-11), "version: 2.0", datetime(1930-01-01), "starting foo", datetime(1953-01-01), "ending foo" ] | parse LogEntry with 'version: ' Version | scan declare (current_version:string) with ( step s1: true => current_version = iff(isnotempty(Version), Version, current_version); ) | project Date, FilledVersion = current_version, OriginalVersion = Version, LogEntry
关键部分解释
scan运算符:逐行遍历数据集,我们声明一个current_version变量来保存最近一次出现的非空版本号。step逻辑:每一行都会执行判断:如果当前行的Version字段非空,就将current_version更新为这个值;否则保留之前的current_version值,以此实现自动向下填充。project输出:最后把填充后的字段(命名为FilledVersion)和原字段放在一起,方便你对比验证结果。
另一种实现思路(基于会话分组)
如果你更习惯用分组类的操作,也可以通过标记版本变更点、分组会话后关联数据来实现:
datatable (Date:datetime, LogEntry:string) [ datetime(1910-06-11), "version: 1.0", datetime(1930-01-01), "starting foo", datetime(1953-01-01), "ending foo", datetime(1910-06-11), "version: 2.0", datetime(1930-01-01), "starting foo", datetime(1953-01-01), "ending foo" ] | parse LogEntry with 'version: ' Version | extend version_change = iff(isnotempty(Version), 1, 0) | accumulate session_id = sum(version_change) | summarize latest_version = arg_max(Date, Version).Version by session_id | join kind=inner ( datatable (Date:datetime, LogEntry:string) [ datetime(1910-06-11), "version: 1.0", datetime(1930-01-01), "starting foo", datetime(1953-01-01), "ending foo", datetime(1910-06-11), "version: 2.0", datetime(1930-01-01), "starting foo", datetime(1953-01-01), "ending foo" ] | parse LogEntry with 'version: ' Version | extend version_change = iff(isnotempty(Version), 1, 0) | accumulate session_id = sum(version_change) ) on session_id | project Date, FilledVersion = latest_version, OriginalVersion = Version, LogEntry | sort by Date
这种方法通过accumulate生成会话ID,把同一个版本下的日志归为一组,再通过关联把版本号填充到组内所有行。不过相比之下,第一种scan的写法更简洁直观,推荐优先使用。
内容的提问来源于stack exchange,提问作者Greg Clinton
相关产品推荐
相关产品推荐

