Python3.9中Pandas concat()函数内报错:无对象可拼接,调试模式正常
问题描述
- 基于Python 3.9开发,多个Pandas DataFrame存储在
self.pointcloud字典中,结构示例为self.pointcloud['20180712']['data'],每个['data']的列结构完全一致。 - 编写了
get_data函数用于筛选指定tag对应的DataFrame列表:
def get_data(self, tag): return [self.pointcloud[pc]['data'] for pc in self.pointcloud if self.pointcloud[pc]['data']['tag'].unique() == [tag]]
- 调试时直接执行
df_list = self.get_data(tag)后调用pd.concat(df_list)能正常运行,但将逻辑封装到merge_pointclouds函数后调用,抛出ValueError: No objects to concatenate错误:
def merge_pointclouds(self, tag): df_list = self.get_data(tag) return pd.concat(df_list)
- 诡异的是,调试模式下进入
merge_pointclouds函数执行完全相同的逻辑却能正常运行,完整报错栈如下:
Traceback (most recent call last): File "C:\Program Files\JetBrains\PyCharm Community Edition 2020.2.2\plugins\python-ce\helpers\pydev\pydevd.py", line 1448, in _exec pydev_imports.execfile(file, globals, locals) # execute the script File "C:\Program Files\JetBrains\PyCharm Community Edition 2020.2.2\plugins\python-ce\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile exec(compile(contents+"\n", file, 'exec'), glob, loc) File "C:/.../PycharmProjects/PCPro2023_0_3_2py39-64/pc.py", line 927, in <module> debug(debugging=debugging) File "C:/.../PycharmProjects/PCPro2023_0_3_2py39-64/pc.py", line 68, in debug pc.set_distance_between() File "C:/.../PycharmProjects/PCPro2023_0_3_2py39-64/pc.py", line 388, in set_distance_between data_top = self.merge_pointclouds(tag='Top') File "C:/.../PycharmProjects/PCPro2023_0_3_py39-64/pc.py", line 221, in merge_pointclouds res = pd.concat(df_list) File "C:\...\PycharmProjects\PCPro2023_0_3_py39-64\venv\lib\site-packages\pandas\util\_decorators.py", line 331, in wrapper return func(*args, **kwargs) File "C:\...\PycharmProjects\PCPro2023_0_3_py39-64\venv\lib\site-packages\pandas\core\reshape\concat.py", line 368, in concat op = _Concatenator( File "C:\...\PycharmProjects\PCPro2023_0_3_py39-64\venv\lib\site-packages\pandas\core\reshape\concat.py", line 425, in __init__ raise ValueError("No objects to concatenate") ValueError: No objects to concatenate
问题原因
get_data的判断逻辑存在隐性错误:self.pointcloud[pc]['data']['tag'].unique()返回的是numpy数组,直接用它和Python列表[tag]做==比较,不会返回单个布尔值,而是返回一个布尔numpy数组。在Python的if判断中,非空的numpy数组会被视为True,但如果数组内容和[tag]不匹配,判断逻辑就会失效,导致筛选不到符合条件的DataFrame,最终df_list为空,触发pd.concat的报错。- 调试与运行时的数据状态不一致:
调试时通常是在self.pointcloud的所有数据完全加载完成后才执行get_data,且可能手动确认了tag的存在;但正常运行时,merge_pointclouds可能在部分数据还没加载完成时就被调用,或者数据加载顺序导致unique()返回的数组状态和调试时不同,最终get_data返回空列表。
修复方案
- 修正
get_data的判断逻辑:- 若需要筛选
tag列只有唯一值且等于目标tag的DataFrame,用np.array_equal比较numpy数组:import numpy as np def get_data(self, tag): target = np.array([tag]) return [self.pointcloud[pc]['data'] for pc in self.pointcloud if np.array_equal(self.pointcloud[pc]['data']['tag'].unique(), target)] - 若只需要筛选
tag列包含目标tag的DataFrame(不管是否有其他tag),可简化判断:def get_data(self, tag): return [self.pointcloud[pc]['data'] for pc in self.pointcloud if tag in self.pointcloud[pc]['data']['tag'].unique()]
- 若需要筛选
- 添加空列表防护:
在merge_pointclouds里先检查df_list是否为空,避免直接调用pd.concat报错:def merge_pointclouds(self, tag): df_list = self.get_data(tag) if not df_list: # 可返回空DataFrame或自定义错误提示 return pd.DataFrame() return pd.concat(df_list) - 确保数据加载完成:
检查调用merge_pointclouds的时机,确保self.pointcloud中的所有数据已经完全加载完毕,避免因数据未就绪导致筛选结果为空。
内容的提问来源于stack exchange,提问作者Spatial Digger
相关产品推荐
相关产品推荐

