Spark-SQL Databricks技术问题:获取Open状态最小日期单条记录
解决方案
要获取Open状态下STRT_DT_EST最小的单条记录,推荐使用Spark-SQL的窗口函数实现,逻辑清晰且高效:
方法一:窗口函数(全局最小Open记录)
适合从指定订单的所有Open记录中,取STRT_DT_EST最小的那一条:
WITH ranked_open_records AS ( SELECT o.svc_ord_nbr AS SVC_ORD_NBR, o.svc_ord_stat_nm AS SVC_ORD_STAT_NM, t.start_date_est AS STRT_DT_EST, t.status_text, -- 按预估开始日期升序排序,最小日期的记录排名为1 ROW_NUMBER() OVER (ORDER BY t.start_date_est ASC) AS rn FROM A o INNER JOIN B t ON t.ticket = o.notif_nbr WHERE o.svc_ord_nbr IN ('021519_574819','110714_246149') AND o.svc_ord_stat_nm = 'Open' ) SELECT SVC_ORD_NBR, SVC_ORD_STAT_NM, STRT_DT_EST, status_text FROM ranked_open_records WHERE rn = 1;
方法二:窗口函数(按订单取最小Open记录)
如果需要每个指定订单各自的最小Open状态记录,只需给窗口函数添加分区:
WITH ranked_open_records AS ( SELECT o.svc_ord_nbr AS SVC_ORD_NBR, o.svc_ord_stat_nm AS SVC_ORD_STAT_NM, t.start_date_est AS STRT_DT_EST, t.status_text, -- 按服务订单分区,每个订单内按预估开始日期升序排序 ROW_NUMBER() OVER (PARTITION BY o.svc_ord_nbr ORDER BY t.start_date_est ASC) AS rn FROM A o INNER JOIN B t ON t.ticket = o.notif_nbr WHERE o.svc_ord_nbr IN ('021519_574819','110714_246149') AND o.svc_ord_stat_nm = 'Open' ) SELECT SVC_ORD_NBR, SVC_ORD_STAT_NM, STRT_DT_EST, status_text FROM ranked_open_records WHERE rn = 1;
方法三:子查询关联
如果不喜欢CTE写法,也可以用子查询先找到最小的STRT_DT_EST,再关联获取完整记录:
SELECT o.svc_ord_nbr AS SVC_ORD_NBR, o.svc_ord_stat_nm AS SVC_ORD_STAT_NM, t.start_date_est AS STRT_DT_EST, t.status_text FROM A o INNER JOIN B t ON t.ticket = o.notif_nbr WHERE o.svc_ord_nbr IN ('021519_574819','110714_246149') AND o.svc_ord_stat_nm = 'Open' AND t.start_date_est = ( SELECT MIN(t2.start_date_est) FROM A o2 INNER JOIN B t2 ON t2.ticket = o2.notif_nbr WHERE o2.svc_ord_nbr IN ('021519_574819','110714_246149') AND o2.svc_ord_stat_nm = 'Open' ) LIMIT 1; -- 若存在多条相同最小日期的记录,用LIMIT 1取单条
说明
原SQL的问题在于:
- 分组包含了
t.status_text,可能拆分Open状态的记录,导致无法直接聚合出最小日期 - 未过滤
Open状态的记录 DISTINCT关键字多余,分组查询本身已经去重
内容的提问来源于stack exchange,提问作者Krishna
相关产品推荐
相关产品推荐

