如何从短语列表生成DataFrame特征列以用于机器学习模型输入
短语集合转One-Hot格式DataFrame实现
核心实现逻辑
- 遍历所有数据行的短语集合,汇总去重得到全量唯一短语列表,作为DataFrame的列名
- 逐行匹配列对应的短语,当前行包含该短语时列值设为1,不包含则设为0
- 最终输出0/1值构成的稀疏结构DataFrame,可直接作为机器学习模型的输入特征
示例输入数据
两组待处理的短语集合示例如下:
# 第1行短语集合 {"TV", "Internet", "Wireless Internet", "Kitchen", "Free Parking on Premises", "Buzzer/Wireless Intercom", "Heating", "Family/Kid Friendly", "Washer,Dryer", "Smoke Detector", "Carbon Monoxide Detector", "First Aid Kit", "Safety Card", "Fire Extinguisher", "Essentials"} # 第2行短语集合 {"TV", "Internet", "Wireless Internet", "Air Conditioning", "Kitchen", "Pets Allowed", "Pets live on this property", "Dog(s)", "Heating", "Family/Kid Friendly", "Washer", "Dryer", "Smoke Detector", "Carbon Monoxide Detector", "Fire Extinguisher", "Essentials", "Shampoo", "Lock on Bedroom Door", "Hangers", "Hair Dryer", "Iron"}
可直接运行的实现代码
import pandas as pd from itertools import chain # 原始短语集合:每个元素对应一行数据的全部短语集合 raw_row_phrase_sets = [ {"TV", "Internet", "Wireless Internet", "Kitchen", "Free Parking on Premises", "Buzzer/Wireless Intercom", "Heating", "Family/Kid Friendly", "Washer,Dryer", "Smoke Detector", "Carbon Monoxide Detector", "First Aid Kit", "Safety Card", "Fire Extinguisher", "Essentials"}, {"TV", "Internet", "Wireless Internet", "Air Conditioning", "Kitchen", "Pets Allowed", "Pets live on this property", "Dog(s)", "Heating", "Family/Kid Friendly", "Washer", "Dryer", "Smoke Detector", "Carbon Monoxide Detector", "Fire Extinguisher", "Essentials", "Shampoo", "Lock on Bedroom Door", "Hangers", "Hair Dryer", "Iron"} ] # 汇总所有行的短语,去重得到全量列名 all_columns = list(set(chain(*raw_row_phrase_sets))) # 逐行生成0/1标记 mark_rows = [] for phrase_set in raw_row_phrase_sets: current_row = [1 if col_name in phrase_set else 0 for col_name in all_columns] mark_rows.append(current_row) # 生成最终DataFrame feature_df = pd.DataFrame(mark_rows, columns=all_columns)
结果说明
生成的feature_df会自动覆盖所有出现过的唯一短语作为列,每一行的取值严格匹配对应行的短语包含情况,完全适配机器学习模型的特征输入要求。
内容的提问来源于stack exchange,提问作者Dave
相关产品推荐
相关产品推荐

