编写函数返回文件1独有的去重单词，标点处理问题求助

我想编写一个函数，读取两个文本文件并返回文件1中存在但文件2中不存在的去重单词。目前遇到的核心问题是单词中的标点会导致程序将其判定为不同单词，例如"little"和"little!"会被视为两个不同的单词。我尝试使用.replace()方法处理但未成功，且本次练习不允许导入任何库。当前返回的结果集仍包含标点，导致结果冗余。

原代码 #

def wsub(fn1, fn2):
    parole_risultato = []
    parole_testo1 = []
    parole_testo2 = []
    with open(fn1, 'r', encoding='UTF-8-SIG') as x:
        testo1 = x
        for linea1 in testo1:
            for parola1 in linea1.split():
                for carattere1 in parola1:
                    if carattere1.isalpha() == False:
                        parola1.replace(carattere1, ' ')
                parola1 = parola1.lower()
                parole_testo1.append(parola1)
    with open(fn2, 'r', encoding='UTF-8-SIG') as y:
        testo2 = y
        for linea2 in testo2:
            for parola2 in linea2.split():
                for carattere2 in parola2:
                    if carattere2.isalpha() == False:
                        parola2.replace(carattere2, ' ')
                parola2 = parola2.lower()
                parole_testo2.append(parola2)
    for elemento1 in parole_testo1:
        if elemento1 not in parole_testo2:
            parole_risultato.append(elemento1)
    return set(parole_risultato), len(set(parole_risultato))

问题根源 #

你用.replace()没生效是因为Python字符串是不可变类型，parola1.replace(...)只会返回新字符串，不会修改原变量。另外，把标点换成空格可能拆分单词，更合理的做法是直接移除非字母字符。

修正后的代码 #

def wsub(fn1, fn2):
    # 清理单词：只保留字母并转小写
    def clean_word(word):
        cleaned_chars = []
        for char in word:
            if char.isalpha():
                cleaned_chars.append(char)
        return ''.join(cleaned_chars).lower()
    
    # 用集合存储处理后的单词，天然去重
    words_file1 = set()
    words_file2 = set()
    
    # 处理文件1
    with open(fn1, 'r', encoding='UTF-8-SIG') as f:
        for line in f:
            for word in line.split():
                cleaned = clean_word(word)
                if cleaned:  # 跳过空字符串（比如全标点的情况）
                    words_file1.add(cleaned)
    
    # 处理文件2
    with open(fn2, 'r', encoding='UTF-8-SIG') as f:
        for line in f:
            for word in line.split():
                cleaned = clean_word(word)
                if cleaned:
                    words_file2.add(cleaned)
    
    # 求文件1独有的单词集合
    unique_words = words_file1 - words_file2
    return unique_words, len(unique_words)

核心优化点 #

• 新增clean_word函数，彻底过滤非字母字符，解决标点导致的单词判定问题
• 用集合存储单词，自动去重，避免后续重复处理，效率更高
• 利用集合差集操作直接得到结果，替代原循环判断，代码更简洁
• 过滤清理后为空的字符串，避免无效数据干扰结果

内容的提问来源于stack exchange，提问作者Valerio Spera