如何按列验证DataFrame值合规性？含自定义布尔验证函数需求

问题场景 #

给定如下Pandas DataFrame和规则字典：

import pandas as pd

df = pd.DataFrame({'A': ['foo', None, '2'],
 'B': ['baz', '1', '4'],
 'C': [None, 'bar', None],
 'D': [None, None, None]})

# DataFrame显示样式：
#      A    B    C   D
# 0  foo  baz  NaN NaN
# 1  NaN    1  bar NaN
# 2    2    4  NaN NaN

dict_ = {
    "A": "CONDTIONNAL",
    "B": "MANDATORY",
    "C": "MANDATORY",
    "D": "OPTIONAL"
}

需求：

1. 生成指定格式的检查结果字符串，显示每列合规状态，缺失值列需标注具体缺失的行索引
2. 编写自定义函数，所有列合规时返回True，否则返回False

规则定义：

• MANDATORY：列中不允许存在任何缺失值
• CONDTIONNAL/OPTIONAL：允许列中存在缺失值

解决方案 #

实现代码 #

import pandas as pd

def check_columns_compliance(df, rules):
    result_lines = []
    all_compliant = True
    
    for col, rule in rules.items():
        # 获取当前列所有缺失值的行索引列表
        null_indices = df[df[col].isnull()].index.tolist()
        
        if rule == "MANDATORY":
            if not null_indices:
                result_lines.append(f"    Column {col} is OK")
            else:
                result_lines.append(f"    Column {col} is not OK / values missing at {null_indices}")
                all_compliant = False
        else:
            # CONDTIONNAL和OPTIONAL规则默认允许缺失值
            result_lines.append(f"    Column {col} is OK")
    
    # 拼接成目标格式的字符串
    result_str = '\n'.join(result_lines)
    return result_str, all_compliant

# 测试调用
df = pd.DataFrame({'A': ['foo', None, '2'],
 'B': ['baz', '1', '4'],
 'C': [None, 'bar', None],
 'D': [None, None, None]})

dict_ = {
    "A": "CONDTIONNAL",
    "B": "MANDATORY",
    "C": "MANDATORY",
    "D": "OPTIONAL"
}

output_str, is_compliant = check_columns_compliance(df, dict_)
# 打印格式化结果
print(f'"""\n{output_str}\n"""')
# 打印整体合规状态
print("所有列是否合规：", is_compliant)

运行输出 #

"""
    Column A is OK
    Column B is OK
    Column C is not OK / values missing at [0, 2]
    Column D is OK
"""
所有列是否合规： False

代码说明 #

• 缺失值索引提取：通过df[df[col].isnull()].index.tolist()精准获取缺失值对应的行索引，解决了仅能判断是否存在缺失的问题
• 规则逻辑处理：针对不同规则分别判断，仅MANDATORY列会检查缺失值并标记不合规状态
• 双返回值设计：同时返回格式化的结果字符串和整体合规状态，满足两种需求

内容的提问来源于stack exchange，提问作者user21474411