求助:Python去除DataFrame列内单元格重复单词的可行方案
去除DataFrame单元格内重复单词的解决方法
方法1:利用Python 3.7+有序字典(推荐)
Python 3.7及以上版本的普通字典会保留键的插入顺序,用dict.fromkeys()可以快速实现去重,代码简洁高效:
import pandas as pd # 示例数据 data = {'words': ['car apple car good', 'good bad well good', 'car apple bus food']} df = pd.DataFrame(data) # 生成去重后的列 df['expected'] = df['words'].str.split().apply(lambda x: ' '.join(dict.fromkeys(x))).str.strip()
方法2:修复你原有的OrderedDict代码
你之前的代码问题出在OrderedDict.fromkeys(x).keys()返回的是odict_keys迭代器对象,str.join无法直接处理,只需将其转为可迭代的字符串序列即可:
from collections import OrderedDict import pandas as pd data = {'words': ['car apple car good', 'good bad well good', 'car apple bus food']} df = pd.DataFrame(data) df['expected'] = df['words'].str.split().apply(lambda x: ' '.join(OrderedDict.fromkeys(x).keys())).str.strip()
方法3:手动遍历去重(兼容性最强)
如果需要兼容低版本Python,或者想要更直观的逻辑,可以手动遍历单词并记录已出现的词汇:
import pandas as pd def remove_duplicates(text): seen = set() unique_words = [] for word in text.split(): if word not in seen: seen.add(word) unique_words.append(word) return ' '.join(unique_words) data = {'words': ['car apple car good', 'good bad well good', 'car apple bus food']} df = pd.DataFrame(data) df['expected'] = df['words'].apply(remove_duplicates)
运行任意一种方法后,都能得到你期望的结果:
| words | expected |
|---|---|
| car apple car good | car apple good |
| good bad well good | good bad well |
| car apple bus food | car apple bus food |
内容的提问来源于stack exchange,提问作者BROFL
相关产品推荐
相关产品推荐

