修复错误（双重）编码CSV中的Mojibake字符乱码

你遇到的是典型的UTF-8编码字符被错误用Latin-1（ISO-8859-1）解码后再次编码导致的Mojibake问题。比如原字符é（UTF-8字节为\xC3\xA9），错误用Latin-1解码会得到é，再存为UTF-8就会显示成Ì©，和你文件里的Fr̩chette对应正确的Fréchette一致。

修复核心逻辑 #

反向还原错误的编码流程：

• 先用错误的编码（Latin-1）读取文件内容，得到错误解码后的字符串
• 将该字符串重新编码为Latin-1，还原出最初的UTF-8字节
• 最后用UTF-8解码这些字节，得到正确的字符

Python 实现示例 #

直接处理整个文件

适合文件体积不大的场景：

# 读取乱码文件，用Latin-1解码
with open("broken_movies.csv", "r", encoding="latin-1") as f:
    content = f.read()

# 还原正确编码
fixed_content = content.encode("latin-1").decode("utf-8")

# 写入修复后的文件
with open("fixed_movies.csv", "w", encoding="utf-8") as f:
    f.write(fixed_content)

逐行处理大文件

避免一次性加载大文件占用过多内存：

with open("broken_movies.csv", "r", encoding="latin-1") as infile, \
     open("fixed_movies.csv", "w", encoding="utf-8") as outfile:
    for line in infile:
        fixed_line = line.encode("latin-1").decode("utf-8")
        outfile.write(fixed_line)

结合CSV模块处理复杂字段

如果CSV存在包含逗号的特殊字段，用csv模块避免破坏文件结构：

import csv

with open("broken_movies.csv", "r", encoding="latin-1") as infile, \
     open("fixed_movies.csv", "w", encoding="utf-8", newline="") as outfile:
    reader = csv.DictReader(infile)
    writer = csv.DictWriter(outfile, fieldnames=reader.fieldnames)
    writer.writeheader()
    for row in reader:
        # 逐个修复字段编码
        fixed_row = {k: v.encode("latin-1").decode("utf-8") for k, v in row.items()}
        writer.writerow(fixed_row)

修复效果验证 #

拿你提供的示例内容测试：

• Dolores VelÌÁzquez → Dolores Velázquez
• Richard Fr̩chette → Richard Fréchette
• Fran̤ois Papineau → François Papineau
• Jean-Philippe C̫t̩ → Jean-Philippe Côté

注意事项 #

• 该方法仅适用于UTF-8→Latin-1→UTF-8的编码错误流程，如果是其他编码混淆（比如GBK与UTF-8），需要调整对应编码参数
• 修复前建议备份原文件，避免操作失误导致数据丢失

内容的提问来源于stack exchange，提问作者LuHo