如何向量化匹配DataFrame的cui列与字典生成symptom列？

给定如下简化版字典：

_d = {
    "pain": 
        ["C0030193",
        "C0150055",
        "C0151825",
        "C0184567"],
    "anxiety": 
        ["C0003467",
        "C0003469",
        "C0027769",
        "C0154587",
        "C0231397",
        "C0231401",
        "C0231402"],
    "depression": 
        ["C0001539",
        "C0005587",
        "C0011579",
        "C0011581",
        "C0024517",
        "C0086132"],
    "fatigue": 
        ["C0015672"]
}

以及如下DataFrame：

import pandas as pd

df = pd.DataFrame({
    "cui": ["C0015672", "C0015634", "C0011579", "C0030193", "C0031193", "C0030193"]
})

需将df的cui列与字典匹配：若CUI值存在于字典的任意值列表中，生成新列symptom并填入对应字典键；否则值为NaN。期望输出：

cui    symptom
0  C0015672     fatigue
1  C0015634        NaN
2  C0011579  depression
3  C0030193        pain
4  C0031193        NaN
5  C0030193        pain

因数据量达数千万行，行遍历方式速度极慢，需向量化解决方案。

核心思路是先反转原字典，构建CUI -> 症状的直接映射，再利用Pandas的向量化方法完成匹配，全程无需遍历行，效率极高。

步骤1：反转字典，构建CUI到症状的映射 #

cui_to_symptom = {}
for symptom, cui_list in _d.items():
    for cui in cui_list:
        cui_to_symptom[cui] = symptom

步骤2：用map()方法生成symptom列 #

df['symptom'] = df['cui'].map(cui_to_symptom)

map()是Pandas的向量化操作，内部采用C级循环处理，比Python层面的行遍历快数个数量级，完全适配千万级数据量。

完整代码示例 #

import pandas as pd

# 原字典
_d = {
    "pain": ["C0030193", "C0150055", "C0151825", "C0184567"],
    "anxiety": ["C0003467", "C0003469", "C0027769", "C0154587", "C0231397", "C0231401", "C0231402"],
    "depression": ["C0001539", "C0005587", "C0011579", "C0011581", "C0024517", "C0086132"],
    "fatigue": ["C0015672"]
}

# 构造DataFrame
df = pd.DataFrame({
    "cui": ["C0015672", "C0015634", "C0011579", "C0030193", "C0031193", "C0030193"]
})

# 反转字典
cui_to_symptom = {}
for symptom, cui_list in _d.items():
    for cui in cui_list:
        cui_to_symptom[cui] = symptom

# 生成symptom列
df['symptom'] = df['cui'].map(cui_to_symptom)

print(df)

输出结果 #

cui    symptom
0  C0015672     fatigue
1  C0015634        NaN
2  C0011579  depression
3  C0030193        pain
4  C0031193        NaN
5  C0030193        pain

内容的提问来源于stack exchange，提问作者horcle_buzz