如何验证Polars DataFrame指定列及类型并处理特殊列名？

问题背景 #

需要验证Polars DataFrame中是否存在指定列及其对应数据类型，允许DataFrame包含额外列。使用Patito进行验证时遇到两个核心问题：

• DataFrame包含多余列时触发ValidationError
• 列名含空格或连字符（如“column 1”“column-1”）时，无法在Patito Model中正常定义

问题复现 #

1. 多余列导致验证失败 #

import polars as pl
import patito as pt

class UserModel(pt.Model):
    name: str
    age: int

df = pl.DataFrame(
    {
        "name": ["Peter", "Anna", "Tyler"],
        "age": [29, 18, 49],
        "location": ["Chicago", "Paris", "Singapore"],
    }
)

UserModel.validate(df)  # 因多余列"location"触发ValidationError

2. 特殊列名无法定义Model #

import polars as pl
import patito as pt

class UserModel(pt.Model):
    name: str
    column-1: int  # 语法错误：Python变量名仅允许字母、数字和下划线

df = pl.DataFrame(
    {
        "name": ["Peter", "Anna", "Tyler"],
        "column-1": [29, 18, 49],
        "location": ["Chicago", "Paris", "Singapore"],
    }
)

UserModel.validate(df)  # 列名定义失败，同时因多余列报错

解决方案 #

1. 允许多余列验证 #

Patito的validate方法默认启用严格模式（strict=True），会检查DataFrame列与Model完全匹配。只需将strict参数设为False，即可仅验证指定列的存在和类型，忽略额外列：

UserModel.validate(df, strict=False)

2. 处理特殊列名 #

Patito基于Pydantic，可通过Field的alias参数将合法的Python变量名映射到特殊格式的列名：

import polars as pl
import patito as pt
from pydantic import Field

class UserModel(pt.Model):
    name: str
    column_1: int = Field(alias="column-1")  # 映射连字符列名
    column_2: str = Field(alias="column 2")  # 映射带空格的列名

df = pl.DataFrame(
    {
        "name": ["Peter", "Anna", "Tyler"],
        "column-1": [29, 18, 49],
        "column 2": ["foo", "bar", "baz"],
        "location": ["Chicago", "Paris", "Singapore"],
    }
)

UserModel.validate(df, strict=False)  # 成功验证，忽略多余列

如果需要批量处理特殊列名（比如统一将连字符转为下划线），可通过Model配置alias_generator实现：

import polars as pl
import patito as pt
from pydantic import AliasGenerator

class UserModel(pt.Model):
    model_config = pt.ConfigDict(
        alias_generator=AliasGenerator(
            alias=lambda var_name: var_name.replace("_", "-"),  # 变量名转列名（下划线→连字符）
            validation_alias=lambda col_name: col_name.replace("-", "_")  # 列名转变量名（连字符→下划线）
        )
    )
    name: str
    column_1: int  # 自动对应列名"column-1"

df = pl.DataFrame(
    {
        "name": ["Peter", "Anna", "Tyler"],
        "column-1": [29, 18, 49],
        "location": ["Chicago", "Paris", "Singapore"],
    }
)

UserModel.validate(df, strict=False)  # 正常验证

内容的提问来源于stack exchange，提问作者M Klingert