如何简化Pandas账户名称与机构部门代码一致性校验函数
问题描述
现有数据集包含ACCOUNT_NAME(账户名称)和INSTITUTIONAL_SECTOR_CODE(机构部门代码)两列,业务校验规则如下:
- 若账户名称包含
list1 = ['limited','Hotel','Company','Restaurant','Sacco']中的元素,对应的机构部门代码需匹配list2 = ['NFCPVT','OLENT','MENT','SENT','MFI']中的对应值 - list1前4个元素对应的账户可匹配list2前4个任意代码
- list1最后一个元素Sacco仅可匹配list2的MFI值
- 最终需要输出不在对应允许值内的机构部门代码列表或计数统计
当前采用硬编码方式实现,现有代码如下:
import pandas as pd import numpy as np def acc_name_and_inst_sec_codes_consistency(df): #account name Comp_acc_name = ['ltdt','limited','Hotel','Company','Restaurant','Consultant','technologies','Service','atelier','quincaillerie'] Sacco_acc_name = ['SACCO'] inst_sector_code_comp_acc_name = ['NFCPVT','OLENT','MENT','SENT','NFCPUB'] inst_sector_code_Sacco_acc_name = ['MFI'] Filtered_comp_acc_name = df[df['ACCOUNT_NAME'].str.contains('|'.join(Comp_acc_name), case=False)] Filtered_sacco_acc_name = df[df['ACCOUNT_NAME'].str.contains('|'.join(Sacco_acc_name), case=False)] print('Below are institutional sector codes that shouldnot be included for the Account name with',\ Comp_acc_name,'in it\n\n',Filtered_comp_acc_name[~Filtered_comp_acc_name['INSTITUTIONAL_SECTOR_CODE'].\ isin(inst_sector_code_comp_acc_name)]['INSTITUTIONAL_SECTOR_CODE_DESC'].value_counts().\ rename_axis('INSTITUTIONAL_SECTOR_CODE_DESC').reset_index(name='counts').sort_values('counts', ascending = False).\ to_string(index=False),'\n\n' 'Below are institutional sector codes that shouldnot be included for the Account name with', Sacco_acc_name,'in it\n\n',Filtered_sacco_acc_name[~Filtered_sacco_acc_name['INSTITUTIONAL_SECTOR_CODE'].\ isin(inst_sector_code_Sacco_acc_name)]['INSTITUTIONAL_SECTOR_CODE_DESC'].value_counts().\ rename_axis('INSTITUTIONAL_SECTOR_CODE_DESC').reset_index(name='counts').sort_values('counts', ascending = False).\ to_string(index=False))
样例输出如下:
Below are institutional sector codes that shouldnot be included for the Account name with ['ltdt', 'limited', 'Hotel', 'Company', 'Restaurant', 'Consultant', 'technologies', 'Service', 'atelier', 'quincaillerie'] in it INSTITUTIONAL_SECTOR_CODE_DESC counts Not Applicable 11012 Individuals 1537 Insurance 9 MFI/SACCO (Deposit Taking) 3 Below are institutional sector codes that shouldnot be included for the Account name with ['SACCO'] in it INSTITUTIONAL_SECTOR_CODE_DESC counts Not Applicable 1001 Individuals 28 Non Financial Companies - Private 1
请问有什么方法可以简化上述函数?
简化方案
核心优化思路是把重复的统计逻辑抽离,同时将匹配规则配置化,避免硬编码带来的维护问题,简化后代码如下:
import pandas as pd import numpy as np # 抽离重复的非法值统计逻辑 def get_invalid_count_str(filtered_df, allowed_codes): invalid_df = filtered_df[~filtered_df['INSTITUTIONAL_SECTOR_CODE'].isin(allowed_codes)] count_res = invalid_df['INSTITUTIONAL_SECTOR_CODE_DESC'].value_counts()\ .rename_axis('INSTITUTIONAL_SECTOR_CODE_DESC')\ .reset_index(name='counts')\ .sort_values('counts', ascending=False) return count_res.to_string(index=False) def acc_name_and_inst_sec_codes_consistency(df): # 规则配置:每个元素为(关键词列表, 允许的代码列表, 规则描述) RULES = [ ( ['ltdt','limited','Hotel','Company','Restaurant','Consultant','technologies','Service','atelier','quincaillerie'], ['NFCPVT','OLENT','MENT','SENT','NFCPUB'], '普通企业类账户关键词' ), ( ['SACCO'], ['MFI'], 'Sacco类账户关键词' ) ] for keywords, allowed_codes, rule_desc in RULES: filtered_df = df[df['ACCOUNT_NAME'].str.contains('|'.join(keywords), case=False)] count_str = get_invalid_count_str(filtered_df, allowed_codes) print(f"以下是账户名称包含{rule_desc}时不允许出现的机构部门代码统计:\n\n{count_str}\n")
优化点说明
- 重复的统计逻辑统一封装为工具函数,后续调整统计格式、排序规则只需要修改一处即可
- 匹配规则用结构化的配置管理,新增校验规则不需要修改主逻辑,直接在RULES列表里加对应条目即可,可维护性大幅提升
- 去掉冗余的代码拼接逻辑,用f-string格式化输出,可读性更强
- 输出内容改为中文,更符合使用习惯,如果需要保留英文输出只需要修改print的模板即可
内容的提问来源于stack exchange,提问作者VMO
相关产品推荐
相关产品推荐

