使用Pandas批量解析CSV文件时遇EmptyDataError问题求助

问题背景 #

用for循环批量解析多个CSV文件并合并到单个工作表，解析几个文件后Pandas持续报错，已尝试指定编码iso-8859-1但无效，报错文件格式和成功解析的类似。

现有代码 #

for file in files:
   data = pd.read_csv(file,encoding='iso-8859-1')
   print(data)
...    

CSV文件示例数据 #

Filesystem      Size  Used Avail Use% Mounted on
0          /dev/sda3       192G   24G  168G  13% /
1  /dev/sda1       512M  7.5M  505M   2% /boot/efi
  Filesystem      Size  Used Avail Use% Mounted on
0          /dev/sda1       200G  101G  100G  51% /
1       /dev/sdd1        50G   33M   50G   1% /u03
2        /dev/sdf1       400G   33M  400G   1% /wa
3       /dev/sde1       300G  230G   71G  77% /edi
4       /dev/sdb1        50G   33M   50G   1% /u01
5      /dev/sdg1        50G   11G   37G  22% /swap
6       /dev/sdc1        50G   33M   50G   1% /u02
  Filesystem      Size  Used Avail Use% Mounted on
0          /dev/sda1       200G   15G  186G   8% /
1       /dev/sdb1       300G  195G  106G  65% /u01
2   /dev/sdc1       100G   71G   30G  71% /opt/IBM
  Filesystem      Size  Used Avail Use% Mounted on
0          /dev/sda1       200G   14G  187G   7% /
1   /dev/sdb1       100G   66G   34G  66% /opt/IBM
2       /dev/sdc1       300G  158G  142G  53% /u01
  Filesystem      Size  Used Avail Use% Mounted on
0          /dev/sda1       200G  9.3G  191G   5% /
1       /dev/sdc1       100G   18G   83G  18% /u01
2   /dev/sdb1       150G   49G  102G  33% /opt/IBM

报错信息 #

Traceback (most recent call last):
  File "<pyshell#95>", line 2, in <module>
    data = pd.read_csv(file,encoding='iso-8859-1')
  File "C:\Users\102712\AppData\Local\Programs\Python\Python311\Lib\site-packages\pandas\util\_decorators.py", line 211, in wrapper
    return func(*args, **kwargs)
  File "C:\Users\102712\AppData\Local\Programs\Python\Python311\Lib\site-packages\pandas\util\_decorators.py", line 331, in wrapper
    return func(*args, **kwargs)
  File "C:\Users\102712\AppData\Local\Programs\Python\Python311\Lib\site-packages\pandas\io\parsers\readers.py", line 950, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "C:\Users\102712\AppData\Local\Programs\Python\Python311\Lib\site-packages\pandas\io\parsers\readers.py", line 605, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "C:\Users\102712\AppData\Local\Programs\Python\Python311\Lib\site-packages\pandas\io\parsers\readers.py", line 1442, in __init__
    self._engine = self._make_engine(f, self.engine)
  File "C:\Users\102712\AppData\Local\Programs\Python\Python311\Lib\site-packages\pandas\io\parsers\readers.py", line 1753, in _make_engine
    return mapping[engine](f, **self.options)
  File "C:\Users\102712\AppData\Local\Programs\Python\Python311\Lib\site-packages\pandas\io\parsers\c_parser_wrapper.py", line 79, in __init__
    self._reader = parsers.TextReader(src, **kwds)
  File "pandas\_libs\parsers.pyx", line 554, in pandas._libs.parsers.TextReader.__cinit__
pandas.errors.EmptyDataError: No columns to parse from file

解决方案 #

1. 过滤空文件并添加异常捕获 #

EmptyDataError通常对应空文件或全空白内容的文件，读取前先判断文件大小，同时捕获异常避免循环中断：

import os
import pandas as pd

for file in files:
    # 跳过空文件
    if os.path.getsize(file) == 0:
        print(f"跳过空文件: {file}")
        continue
    # 捕获解析异常
    try:
        data = pd.read_csv(file, encoding='iso-8859-1', delim_whitespace=True)
        print(data)
        # 这里添加数据合并逻辑
    except pd.errors.EmptyDataError:
        print(f"文件无有效数据: {file}")

2. 指定空格分隔符适配数据格式 #

你的数据实际是多空格分隔的表格，不是标准逗号分隔CSV，添加delim_whitespace=True参数让Pandas自动识别空格作为分隔符，兼容多空格情况。

3. 清理表头多余空格 #

部分文件表头前有多余空格，读取后可统一清理：

data.columns = data.columns.str.strip()

内容的提问来源于stack exchange，提问作者soumya sahu