Python Pandas根据其他列值替换指定列为NaN的实现及问题排查
问题背景
现有如下测试DataFrame:
| contact | phone1_x | phone2_x | phone1_y | phone2_y | Match1 | Match2 |
|---|---|---|---|---|---|---|
| 1 | 1234 | 12 | 1234 | True | False | |
| 2 | 12345 | 123 | 123 | False | True | |
| 33 | 4444 | 22 | 2343321 | 3223455 | False | False |
| 66 | 55555 | 333 | 55555 | 333 | True | True |
需求说明
Match1为True时将phone1_y值置空为NaN,Match2为True时将phone2_y值置空,两种期望输出如下:
第一种期望输出
| contact | phone1_x | phone2_x | phone1_y | phone2_y | Match1 | Match2 |
|---|---|---|---|---|---|---|
| 1 | 1234 | 12 | True | False | ||
| 2 | 12345 | 123 | False | True | ||
| 33 | 4444 | 22 | 2343321 | 3223455 | False | False |
| 66 | 55555 | 333 | True | True |
第二种期望输出
| contact | phone1_x | phone2_x | phone1_y | phone2_y | Match1 | Match2 |
|---|---|---|---|---|---|---|
| 1 | 1234 | 12 | 12 | True | False | |
| 2 | 12345 | 123 | 12345 | False | True | |
| 33 | 4444 | 22 | 2343321 | 3223455 | False | False |
| 66 | 55555 | 333 | True | True |
问题排查
使用Python Pandas开发,初始编写代码如下:
df = pd.read_csv('TM.csv', sep=',') #x df1 = pd.read_csv('TM_verrijk.csv', sep=',') #y df['phone1'] = df['phone1'].str.replace('-', '') df['phone1'] = df['phone1'].str.replace('+', '') df1['phone1'] = df1['phone1'].str.replace('-', '') df1['phone1'] = df1['phone1'].str.replace('+', '') df['phone2'] = df['phone2'].str.replace('-', '') df['phone2'] = df['phone2'].str.replace('+', '') df1['phone2'] = df1['phone2'].str.replace('-', '') df1['phone2'] = df1['phone2'].str.replace('+', '') dff = df.merge(df1, on='contact') dff['phone1_x'] = pd.to_numeric(dff['phone1_x'], errors='coerce') dff['phone1_y'] = pd.to_numeric(dff['phone1_y'], errors='coerce') dff['phone2_x'] = pd.to_numeric(dff['phone2_x'], errors='coerce') dff['phone2_y'] = pd.to_numeric(dff['phone2_y'], errors='coerce') dff['Match?'] = np.where(dff['phone1_x'] == dff['phone1_y'], 'True', 'False') dff['Match2?'] = np.where(dff['phone2_x'] == dff['phone2_y'], 'True', 'False')
运行后得到的dff和测试DataFrame结构一致,再执行如下代码时出现异常:
dff[['phone1_y','phone2_y']] = np.where(dff[['Match?','Match2?']], np.nan, dff[['phone1_x','phone2_x']])
运行后所有值都被替换为NaN。
根因与解决方案
最终定位问题为Match?和Match2?是字符串类型,非布尔类型,np.where无法正确识别判断条件,转为布尔类型即可正常运行,修复代码如下:
dff['Match?'] = dff['Match?'].map({'False':False, 'True':True}) dff['Match2?'] = dff['Match2?'].map({'False':False, 'True':True})
内容的提问来源于stack exchange,提问作者hii
相关产品推荐
相关产品推荐

