Python如何基于带上下限的查找表自动完成变量分箱及WoE映射
WoE值自动映射实现问题
编辑:该问题被标记为重复问题,但我参考了另一问题的解决方案后发现其无法解决我的需求,因此本问题不属于重复问题。
我有一个包含数值型变量的数据集,需要将这些变量映射到查找表中存储的Weight of Evidence(WoE)值。下文是示例,但我有大量变量需要进行转换,请问如何实现自动化处理?我之前用SAS和SQL实现过该需求,但不知道Python中要如何操作。我在网上检索到的案例大多使用pd.cut,但该方法需要手动编码分箱区间,我不清楚如何基于已有的查找表自动完成分箱,要求分箱规则为下限大于等于(>=)、上限小于(<)。
我编写了如下代码,但在mask语句处报错,麻烦帮我指出代码存在的问题:
temp_dict = {'var1': [4, 2, 2, 4, 7, 4, 12, 8, 17, 42, 33, 10, 7, 5, 17, 29, 64, 20, 39, 6], 'var2': [9, 8, 2, 17, 14, 14, 16, 8, -999999999,-999999999, 10, -999999999, 16, 20, 17, -999999999, -999999999, -999999999,-999999999, 3]} df = pd.DataFrame.from_dict(temp_dict)
查找表规则
| Index | feature_name | lower_limit | upper_limit | WoE |
|---|---|---|---|---|
| 0 | var1 | -999999990 | 1.5 | 1.363071 |
| 1 | var1 | 1.5 | 2.5 | 1.231135 |
| 2 | var1 | 2.5 | 3.5 | 0.819431 |
| 3 | var1 | 3.5 | 14.5 | 0.617399 |
| 4 | var1 | 14.5 | 15.5 | -0.62779 |
| 5 | var1 | 15.5 | 20.5 | -0.706259 |
| 6 | var1 | 20.5 | 999999990 | -0.950134 |
| 7 | var1 | -999999999 | -999999999 | 0 |
| 8 | var1 | NaN | NaN | 0 |
| 0 | var2 | -999999990 | 0.5 | 0.951202 |
| 1 | var2 | 0.5 | 1.5 | 0.427215 |
| 2 | var2 | 1.5 | 2.5 | 0.218994 |
| 3 | var2 | 2.5 | 3.5 | 0.026722 |
| 4 | var2 | 3.5 | 13.5 | -0.071878 |
| 5 | var2 | 13.5 | 999999990 | -1.615368 |
| 6 | var2 | -999999999 | -999999999 | 0 |
| 7 | var2 | NaN | NaN | 0 |
期望输出示例
| var1 | var2 | WoE_var1 | WoE_var2 |
|---|---|---|---|
| 4 | 9 | 0.617399 | -0.071878 |
| 2 | 8 | 1.231135 | -0.071878 |
| 2 | 2 | 1.231135 | 0.218994 |
| 4 | 17 | 0.617399 | -1.615368 |
| 7 | 14 | 0.617399 | -1.615368 |
可行实现方案
提问者后续已找到可运行的实现代码:
binned_vars= ['var1','var2','varn'] short_list= ['var1','var2'] for variable in binned_vars: if variable in short_list: for i in lookup_table[(lookup_table['feature_name'] == variable)]['Index'].to_list(): lower_limit = lookup_table[(lookup_table['feature_name'] == variable) & (lookup_table['Index'] == i)]['lower_limit'].to_list()[0] upper_limit = lookup_table[(lookup_table['feature_name'] == variable) & (lookup_table['Index'] == i)]['upper_limit'].to_list()[0] WoE = lookup_table[(lookup_table['feature_name'] == variable) & (lookup_table['Index'] == i)]['WoE'].to_list()[0] mask = ((df[variable] >= lower_limit) & (df[variable] < upper_limit)) var_name = variable + '_WoE' df.loc[mask,var_name] = WoE
内容的提问来源于stack exchange,提问作者Zenvega
相关产品推荐
相关产品推荐

