使用Polars的is_between过滤日期列时遇Datetime与Time类型匹配错误

环境信息 #

Polars: 0.16.2
Python: 3.11.1
Windows 10

问题描述 #

尝试通过is_between()方法按时间范围过滤Datetime类型的列时，触发PanicException错误，提示无法转换Datetime和Time类型。

复现代码 #

import polars as pl
from datetime import datetime, time

df = pl.date_range(low=datetime(2023, 2, 7), high=datetime(2023, 2, 8), interval="30m", name="date").to_frame()

# 触发错误的过滤代码
df.filter(
    pl.col('date').is_between(time(9, 30), time(14, 30))
)

报错信息 #

PanicException                            Traceback (most recent call last)
Cell In[11], line 1
----> 1 df.filter(
      2     pl.col('date').is_between(time(9, 30, 0, 0), time(14, 30, 0, 0))
      3 )

File d:\My_Path\venv\Lib\site-packages\polars\internals\dataframe\frame.py:2747, in DataFrame.filter(self, predicate)
   2741 if _check_for_numpy(predicate) and isinstance(predicate, np.ndarray):
   2742     predicate = pli.Series(predicate)
   2744 return (
   2745     self.lazy()
   2746     .filter(predicate)  # type: ignore[arg-type]
-> 2747     .collect(no_optimization=True)
   2748 )

File d:\My_Path\venv\Lib\site-packages\polars\internals\lazyframe\frame.py:1146, in LazyFrame.collect(self, type_coercion, predicate_pushdown, projection_pushdown, simplify_expression, no_optimization, slice_pushdown, common_subplan_elimination, streaming)
   1135     common_subplan_elimination = False
   1137 ldf = self._ldf.optimization_toggle(
   1138     type_coercion,
   1139     predicate_pushdown,
   (...)
   1144     streaming,
   1145 )
-> 1146 return pli.wrap_df(ldf.collect())

PanicException: cannot coerce datatypes: ComputeError(Owned("Failed to determine supertype of Datetime(Microseconds, None) and Time"))

解决方案 #

错误根源是is_between要求比较两端的类型必须匹配：目标列是Datetime类型（包含日期+时间），但传入的是Time类型（仅时间），类型不匹配导致转换失败。

只需先从Datetime列中提取时间部分，再进行范围比较即可：

import polars as pl
from datetime import datetime, time

df = pl.date_range(low=datetime(2023, 2, 7), high=datetime(2023, 2, 8), interval="30m", name="date").to_frame()

# 提取时间后执行过滤
df.filter(
    pl.col('date').dt.time().is_between(time(9, 30), time(14, 30))
)

通过.dt.time()方法将Datetime列转换为Time类型，此时与传入的time对象类型一致，就能正常完成is_between过滤。

内容的提问来源于stack exchange，提问作者Dan L