如何用Pydantic 2.11.7捕获Pandas 2.3.0中的缺失NaN值？

前置条件 #

• Python 3.11.7
• Pandas 2.3.0
• Numpy 2.1.3
• Pydantic 2.11.7

Pandas文档说明，数值类型的缺失值会用numpy.nan填充：

In [1]:
import pandas as pd
import numpy as np

In [2]:
s = pd.Series([1, 2], dtype=np.int64).reindex([0, 1, 2])

In [3]:
s
Out[3]: 
0    1.0
1    2.0
2    NaN
dtype: float64

理论上在Pydantic模型中，可以用Literal[numpy.nan]捕获numpy.nan，结合typing.Union就能创建支持复杂逻辑的模型，比如接受正整数或numpy.nan：

In [4]:
import numpy as np
import pydantic

from typing import Union, Literal

class Col(pydantic.BaseModel):
    precision: Union[
        pydantic.conint(ge=1),
        Literal[np.nan]
    ]

但在上述Pandas示例中，遍历Series的values遇到缺失值时，Pydantic验证失败：

In [5]:
for i in s.values:
    print(i)
    Col(precision=i)
Out[5]:
1.0
2.0
nan

---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
Cell In[5], line 3
      1 for i in s.values:
      2     print(i)
----> 3     Col(precision=i)

File ~/........./python3.11/site-packages/pydantic/main.py:253, in BaseModel.__init__(self, **data)
    251 # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks
    252 __tracebackhide__ = True
--> 253 validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
    254 if self is not validated_self:
    255     warnings.warn(
    256         'A custom validator is returning a value other than `self`.
'
    257         "Returning anything other than `self` from a top level model validator isn't supported when validating via `__init__`.
"
    258         'See the `model_validator` docs for more details.',
    259         stacklevel=2,
    260     )

ValidationError: 2 validation errors for Col
precision.constrained-int
  Input should be a finite number [type=finite_number, input_value=nan, input_type=float64]
precision.literal[nan]
  Input should be nan [type=literal_error, input_value=nan, input_type=float64]

但直接传入常规numpy.nan时，验证却能正常通过：

In [6]:
Col(precision=np.nan)
Out[6]:
Col(precision=nan)

这似乎说明Pandas填充的缺失值并非常规numpy.nan，请问该缺失值是什么类型？如何用Pydantic和类型注解优雅捕获它？

目前我有一个临时解决方案：用pydantic.confloat(allow_inf_nan=True)替代Literal[np.nan]，再添加模型验证器确保非NaN值符合要求，但该方案不够优雅：

In [7]:
import math

class Col2(pydantic.BaseModel):
    precision: Union[
        pydantic.conint(ge=1),
        pydantic.confloat(allow_inf_nan=True)
    ]

    @pydantic.model_validator(mode="after")
    def validate_nans(self):
        if self.precision is not None and isinstance(self.precision, float):
            if self.precision.is_integer():
                assert self.precision >= 0
            else:
                assert math.isnan(self.precision)

该方案对多种NaN类型均有效：

In [8]:
import pytest

bad_nan = s.iloc[-1]

test_values = [
    (0, True), (-1, False), (-1.234, False), (1.234, False), (2, True),
    (3.0, True), (-3.0, False),
    (bad_nan, True), (np.nan, True), (float('nan'), True), (np.float64('nan'), True),
    (False, True), (True, True)  # Bool False is 0, True is 1. This is fine.
]

for val, should_pass in test_values:
    if should_pass:
        Col2(precision=val)
    else:
        with pytest.raises(pydantic.ValidationError):
            Col2(precision=val)
Out[8]:
============================== 1 passed in 0.05s ===============================

以下是我尝试过但无效的方案：

• Annotated[pydantic.conint(ge=0), pydantic.AllowInfNan()]
  失败原因：无法在一个注解类型中混合int和NaN浮点类型。
• pydantic.confloat(ge=0, multiple_of=1, allow_inf_nan=True)
  失败原因：无法同时指定ge参数与allow_inf_nan=True（NaN不满足大于等于0的条件）。

内容的提问来源于stack exchange，提问作者MikeFenton