在PyCharm中用Pandas读取CSV文件出现UnicodeDecodeError错误求助

使用环境 #

PyCharm 2022.2.3、Python 3.11、Mac系统

代码示例 #

import pandas as pd
import numpy as np
import plotly.express as px
import plotly.graph_objects as go
import plotly.io as pio
import os

print(os.getcwd())
os.chdir("//Users//apple//Desktop//Employedata")
print(os.getcwd())
pio.templates.default = "plotly_white"
data = pd.read_csv("/Users/apple/Desktop/Employedata/EmployeeSampleData.csv")
print(data)

错误信息 #

/Users/apple/PycharmProjects/pythonProject4/venv/bin/python /Users/apple/PycharmProjects/pythonProject4/main.py 
/Users/apple/PycharmProjects/pythonProject4
/Users/apple/Desktop/Employedata

Traceback (most recent call last):
File "/Users/apple/PycharmProjects/pythonProject4/main.py", line 12, in <module>
data = pd.read_csv("/Users/apple/Desktop/Employedata/EmployeeSampleData.csv")
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/apple/PycharmProjects/pythonProject4/venv/lib/python3.11/site-packages/pandas/util/_decorators.py", line 211, in wrapper
return func(*args, **kwargs)
    ^^^^^^^^^^^^^^^^^^^^^
File "/Users/apple/PycharmProjects/pythonProject4/venv/lib/python3.11/site-packages/pandas/util/_decorators.py", line 331, in wrapper
  return func(*args, **kwargs)
    ^^^^^^^^^^^^^^^^^^^^^
File "/Users/apple/PycharmProjects/pythonProject4/venv/lib/python3.11/site-packages/pandas/io/parsers/readers.py", line 950, in read_csv
  return _read(filepath_or_buffer, kwds)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/apple/PycharmProjects/pythonProject4/venv/lib/python3.11/site-packages/pandas/io/parsers/readers.py", line 605, in _read
  parser = TextFileReader(filepath_or_buffer, **kwds)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/apple/PycharmProjects/pythonProject4/venv/lib/python3.11/site-packages/pandas/io/parsers/readers.py", line 1442, in __init__
  self._engine = self._make_engine(f, self.engine)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/apple/PycharmProjects/pythonProject4/venv/lib/python3.11/site-packages/pandas/io/parsers/readers.py", line 1753, in _make_engine
return mapping[engine](f, **self.options)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/apple/PycharmProjects/pythonProject4/venv/lib/python3.11/site-packages/pandas/io/parsers/c_parser_wrapper.py", line 79, in __init__
self._reader = parsers.TextReader(src, **kwds)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "pandas/_libs/parsers.pyx", line 547, in pandas._libs.parsers.TextReader.__cinit__
File "pandas/_libs/parsers.pyx", line 636, in pandas._libs.parsers.TextReader._get_header
File "pandas/_libs/parsers.pyx", line 852, in pandas._libs.parsers.TextReader._tokenize_rows
File "pandas/_libs/parsers.pyx", line 1965, in pandas._libs.parsers.raise_parser_error
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa0 in position 14194: invalid start byte

Process finished with exit code 1

解决方案 #

• 指定替代编码格式
  错误中的0xa0是不间断空格字符，常见于latin-1或cp1252编码，直接在read_csv中指定对应编码即可：

  # 用latin-1编码读取
  data = pd.read_csv("/Users/apple/Desktop/Employedata/EmployeeSampleData.csv", encoding='latin-1')
  
  # 或用cp1252编码读取
  data = pd.read_csv("/Users/apple/Desktop/Employedata/EmployeeSampleData.csv", encoding='cp1252')
  
  # 若文件是带BOM的UTF-8，尝试utf-8-sig
  data = pd.read_csv("/Users/apple/Desktop/Employedata/EmployeeSampleData.csv", encoding='utf-8-sig')
  

• 检测文件实际编码
  如果不确定文件编码，可使用chardet库检测：

  1. 先安装chardet：

     pip install chardet
     

  2. 编写检测代码：

     import chardet
     with open("/Users/apple/Desktop/Employedata/EmployeeSampleData.csv", 'rb') as f:
         result = chardet.detect(f.read())
     print(f"检测到的编码：{result['encoding']}")
     

  3. 使用检测出的编码值替换read_csv中的encoding参数。

• 临时忽略/替换错误字节
  若只是临时查看数据，可添加errors参数跳过或替换无法解码的字节，但可能丢失部分信息：

  # 替换无法解码的字节为�
  data = pd.read_csv("/Users/apple/Desktop/Employedata/EmployeeSampleData.csv", errors='replace')
  
  # 直接忽略无法解码的字节
  data = pd.read_csv("/Users/apple/Desktop/Employedata/EmployeeSampleData.csv", errors='ignore')
  

内容的提问来源于stack exchange，提问作者askit