使用Numpy dtype读取二进制文件到Pandas DataFrame后删行报错求助

用Numpy.dtype模板读取二进制文件生成Pandas DataFrame后，尝试删除行时触发TypeError: void() takes at least 1 positional argument (0 given)错误，IDE变量查看器查看列名时也会出现相同错误，推测是数据导入方式导致的类型兼容问题。

数据加载代码 #

import numpy as np
import pandas as pd

data_template = np.dtype([
    ('header_a','V22'),
    ('variable_A','>u2'),
    ('gpssec','>u4')
])

with open(source_file, 'rb') as f: 
    byte_data = f.read()
np_data = np.frombuffer(byte_data, data_template)
df = pd.DataFrame(np_data)

触发错误的操作 #

执行行筛选操作时触发报错：

df = df[df['gpssec'] > 1000]

报错栈 #

File C:\ProgramData\anaconda311\Lib\site-packages\pandas\core\frame.py:3798 in __getitem__
  return self._getitem_bool_array(key)

File C:\ProgramData\anaconda311\Lib\site-packages\pandas\core\frame.py:3853 in _getitem_bool_array
  return self._take_with_is_copy(indexer, axis=0)

File C:\ProgramData\anaconda311\Lib\site-packages\pandas\core\generic.py:3902 in _take_with_is_copy
  result = self._take(indices=indices, axis=axis)

File C:\ProgramData\anaconda311\Lib\site-packages\pandas\core\generic.py:3886 in _take
  new_data = self._mgr.take(

File C:\ProgramData\anaconda311\Lib\site-packages\pandas\core\internals\managers.py:978 in take
  return self.reindex_indexer(

File C:\ProgramData\anaconda311\Lib\site-packages\pandas\core\internals\managers.py:751 in  reindex_indexer
  new_blocks = [

File C:\ProgramData\anaconda311\Lib\site-packages\pandas\core\internals\managers.py:752 in <listcomp>
  blk.take_nd(

File C:\ProgramData\anaconda311\Lib\site-packages\pandas\core\internals\blocks.py:880 in take_nd
  new_values = algos.take_nd(

File C:\ProgramData\anaconda311\Lib\site-packages\pandas\core\array_algos\take.py:117 in take_nd
  return _take_nd_ndarray(arr, indexer, axis, fill_value, allow_fill)

File C:\ProgramData\anaconda311\Lib\site-packages\pandas\core\array_algos\take.py:134 in _take_nd_ndarray
  dtype, fill_value, mask_info = _take_preprocess_indexer_and_fill_value(

File C:\ProgramData\anaconda311\Lib\site-packages\pandas\core\array_algos\take.py:582 in _take_preprocess_indexer_and_fill_value
  dtype, fill_value = arr.dtype, arr.dtype.type()

TypeError: void() takes at least 1 positional argument (0 given)

临时解决方法 #

目前只能将相关列数据复制到无损坏列名的空DataFrame中，但操作繁琐，寻求更优方案。

根本原因与解决方案 #

原因 #

问题出在'V22'类型的header_a字段上：这是Numpy的void类型，Pandas在处理void类型数组时，会尝试生成默认填充值arr.dtype.type()，但void类型的构造函数必须传入参数，因此触发报错。

方案1：转换void类型字段格式 #

把header_a字段从void类型转为字节字符串或Unicode字符串，避免Pandas处理void类型：

# 转为字节字符串
data_template = np.dtype([
    ('header_a','S22'),
    ('variable_A','>u2'),
    ('gpssec','>u4')
])

# 或转为Unicode字符串
data_template = np.dtype([
    ('header_a','U22'),
    ('variable_A','>u2'),
    ('gpssec','>u4')
])

如果不需要header_a字段，也可以直接删除该列：

df = pd.DataFrame(np_data).drop('header_a', axis=1)

方案2：使用Pandas from_records方法 #

改用pd.DataFrame.from_records生成DataFrame，该方法会自动将void类型转为object类型的字节数组，避免填充值构造错误：

df = pd.DataFrame.from_records(np_data)

内容的提问来源于stack exchange，提问作者B F