如何实现字典值列表与Pandas列的精准关键词匹配？

需求说明 #

现有含news_text列的DataFrame df，以及一个分类字典（键为分类名，值为对应关键词列表），需要完成：

• 精准匹配新闻文本中的关键词
• 将匹配到的分类名以列表形式存入新列mapped_category
• 无匹配时赋值为NA

解决代码 #

首先准备示例数据：

import pandas as pd
import re

# 构造示例DataFrame
data = {
    'news_text': [
        'lebron james is the great basketball player.',
        'leonardo di caprio has won the oscar for best actor',
        'avatar was directed by steven speilberg.',
        'ronaldo has resigned from manchester united.',
        'argentina beats france in fifa world cup 2022.',
        'joe biden has won the president elections.',
        '2026 fifa WC will be host by canada,mexico and usa combined.'
    ]
}
df = pd.DataFrame(data)

# 构造示例分类字典
category_dict = {
    'category_1': ['lebron james', 'oscar', 'leonardo dicaprio'],
    'category_2': ['basketball', 'steven speilberg','manchester united'],
    'category_3': ['ronaldo', 'argentina','world cup']
}

接下来处理字典并实现匹配逻辑：

# 1. 反转字典：将关键词映射到对应的分类，同时统一为小写（避免大小写不匹配）
keyword_to_category = {}
for category, keywords in category_dict.items():
    for kw in keywords:
        keyword_to_category[kw.lower()] = category

# 2. 定义精准匹配函数：匹配完整短语，避免部分匹配
def get_matched_categories(text):
    text_lower = text.lower()
    matched = []
    for kw, cat in keyword_to_category.items():
        # 使用正则匹配完整短语：前后为空格/文本首尾，避免部分匹配
        pattern = re.compile(r'(?:^|\s)' + re.escape(kw) + r'(?:\s|$)')
        if pattern.search(text_lower):
            matched.append(cat)
    # 无匹配返回NA，否则返回匹配的分类列表
    return matched if matched else pd.NA

# 3. 应用函数生成新列
df['mapped_category'] = df['news_text'].apply(get_matched_categories)

输出结果 #

运行后得到的df如下：

news_text       mapped_category
0          lebron james is the great basketball player.  [category_1, category_2]
1  leonardo di caprio has won the oscar for best actor  [category_1, category_1]
2                avatar was directed by steven speilberg.           [category_2]
3            ronaldo has resigned from manchester united.  [category_3, category_2]
4         argentina beats france in fifa world cup 2022.  [category_3, category_3]
5              joe biden has won the president elections.                    <NA>
6  2026 fifa WC will be host by canada,mexico and ...                    <NA>

说明 #

• 正则匹配确保关键词是完整短语，不会出现类似"world"匹配"world cup"的部分匹配情况
• 统一转为小写处理，避免大小写差异导致的匹配失败
• 针对大型字典也能高效处理，遍历逻辑清晰直接

内容的提问来源于stack exchange，提问作者Bits