Python读取CSV触发UnicodeDecodeError，咨询错误原因

尝试的代码 #

# Import the dataset 
!wget https://research.aalto.fi/files/16859732/urlset.csv.zip

# import the necessary libraries

import zipfile
import pandas as pd
import numpy as np

import io

path_to_zip_file = "/content/urlset.csv.zip"
directory_to_extract_to= "/content/"

data_url = '/content/urlset.csv'

with zipfile.ZipFile(path_to_zip_file, 'r') as zip_ref:
    zip_ref.extractall(directory_to_extract_to)

with io.open(data_url, 'r', encoding='windows-1252') as f:
    data = pd.read_csv(f)

遇到的错误 #

---------------------------------------------------------------------------
UnicodeDecodeError                        Traceback (most recent call last)
<ipython-input-13-4360b73cdc0d> in <module>
      1 with io.open(data_url, 'r', encoding='windows-1252') as f:
----> 2     data = pd.read_csv(f)

8 frames

/usr/local/lib/python3.8/dist-packages/pandas/io/parsers/c_parser_wrapper.py in read(self, nrows)
    222         try:
    223             if self.low_memory:
--> 224                 chunks = self._reader.read_low_memory(nrows)
    225                 # destructive to chunks
    226                 data = _concatenate_chunks(chunks)

/usr/local/lib/python3.8/dist-packages/pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader.read_low_memory()

/usr/local/lib/python3.8/dist-packages/pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._read_rows()

/usr/local/lib/python3.8/dist-packages/pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._tokenize_rows()

/usr/local/lib/python3.8/dist-packages/pandas/_libs/parsers.pyx in pandas._libs.parsers.raise_parser_error()

UnicodeDecodeError: 'charmap' codec can't decode byte 0x8f in position 131198: character maps to <undefined> 

错误原因 #

这个错误的核心问题是：你指定的windows-1252编码无法解析文件里的部分字节（比如报错里的0x8f），说明该CSV文件的实际编码和你设定的不一致。

具体细节：

• windows-1252是单字节编码，仅覆盖了有限的字符范围，文件中存在该编码表未定义的字符，直接导致解码失败。
• 大概率是文件实际采用了其他编码（比如utf-8、latin-1），或者文件中混入了少量非文本的二进制数据。

内容的提问来源于stack exchange，提问作者MarMarhoun