如何基于标题用预定义关键词对DataFrame中能源项目分类？

核心思路 #

不用isin方法（它无法处理复杂逻辑组合和通配符需求），而是为每个主题编写带逻辑判断的匹配函数，结合正则表达式处理通配符，最后对每行标题应用这些函数，收集所有符合的主题编号。

代码实现 #

1. 准备示例数据

import pandas as pd

# 示例数据
data = {
    'title': [
        'Integral Use of Biomass in Electric Power Generation and Use of Heat to Increase Energy Efficiency',
        'simulation of power plant based on nuclear fusion',
        'new industrial processes in the oil sector to increase energy efficiency'
    ]
}
df = pd.DataFrame(data)

2. 定义主题匹配函数

预编译正则表达式提升批量处理效率，同时消除大小写敏感问题：

import re

# 预编译正则（忽略大小写）
topic1_related_re = re.compile(r'(procedure|process|equipment|system|motor)\w*', re.IGNORECASE)
topic3_bio_re = re.compile(r'(biofuel|biomass)\w*', re.IGNORECASE)

def check_topic1(title):
    # 能源效率：efficiency AND (procedure* OR process* OR equipment* OR system* OR motor*)
    title_lower = title.lower()
    return 'efficiency' in title_lower and topic1_related_re.search(title) is not None

def check_topic2(title):
    # 化石燃料：oil OR (natural AND gas) OR LNG OR carbon
    title_lower = title.lower()
    if 'oil' in title_lower or 'lng' in title_lower or 'carbon' in title_lower:
        return True
    return 'natural' in title_lower and 'gas' in title_lower

def check_topic3(title):
    # 可再生能源：(solar AND energy) OR (wind AND energy) OR (renewable AND energy) OR biofuel* OR biomass*
    title_lower = title.lower()
    if ('solar' in title_lower and 'energy' in title_lower) or \
       ('wind' in title_lower and 'energy' in title_lower) or \
       ('renewable' in title_lower and 'energy' in title_lower):
        return True
    return topic3_bio_re.search(title) is not None

def check_topic4(title):
    # 核能：nuclear AND power AND plant
    title_lower = title.lower()
    return 'nuclear' in title_lower and 'power' in title_lower and 'plant' in title_lower

3. 生成主题标注列

# 按主题顺序存入检查函数列表
topic_checks = [check_topic1, check_topic2, check_topic3, check_topic4]

# 遍历每行标题，收集符合的主题编号
df['topic'] = df['title'].apply(
    lambda x: [i+1 for i, check_func in enumerate(topic_checks) if check_func(x)]
)

4. 查看结果

print(df)

输出结果：

title    topic
0  Integral Use of Biomass in Electric Power Gene...  [1, 3]
1  simulation of power plant based on nuclear fusion     [4]
2  new industrial processes in the oil sector to ...  [1, 2]

优化说明 #

• 预编译正则表达式：避免重复编译逻辑，处理大量数据时可显著提升速度
• 统一转小写：彻底消除大小写对匹配结果的干扰
• 拆分逻辑判断：用Python布尔运算实现复杂的AND/OR规则，比纯正则表达式更易读、易维护

内容的提问来源于stack exchange，提问作者Barbara Bressan Rocha