Pandas合并DataFrame报错:TypeError: iterable与itertools.imap类型问题
type object argument after * must be an iterable, not itertools.imap 问题重现
我尝试合并两个各含3列的DataFrame,之前做过类似操作都正常,但这次执行代码时抛出了类型错误。先看一下两个DataFrame的基本信息:
两个DataFrame的info()输出
# pio_smp2_sm.info() <class 'pandas.core.frame.DataFrame'> Int64Index: 12779 entries, 15 to 68311 Data columns (total 3 columns): entityId 12779 non-null object targetEntityId 12779 non-null object eventTime 12779 non-null object dtypes: object(3) memory usage: 399.3+ KB # cm_smp2.info() <class 'pandas.core.frame.DataFrame'> Int64Index: 28035 entries, 40 to 698858 Data columns (total 3 columns): user_id 28035 non-null object product_id 28035 non-null object time_stamp 28035 non-null object dtypes: object(3) memory usage: 876.1+ KB
执行的合并代码
comp_df2 = pd.merge( pio_smp2_sm, cm_smp2, how='inner', left_on=['entityId','targetEntityId'], right_on=['user_id','product_id'] )
触发的错误栈
TypeErrorTraceback (most recent call last) <ipython-input-235-6882a22fe6a1> in <module>() 23 24 ---> 25 comp_df2=pd.merge(pio_smp2_sm,cm_smp2,how='inner',left_on=['entityId','targetEntityId'],right_on=['user_id','product_id']) 26 27 # print(comp_df2.shape[0]) /data2/user/anaconda2/lib/python2.7/site-packages/pandas/core/reshape/merge.pyc in merge(left, right, how, on, left_on, right_on, left_index, right_index, sort, suffixes, copy, indicator, validate) 56 copy=copy, indicator=indicator, 57 validate=validate) ---> 58 return op.get_result() 59 60 /data2/user/anaconda2/lib/python2.7/site-packages/pandas/core/reshape/merge.pyc in get_result(self) 580 self.left, self.right) 581 ---> 582 join_index, left_indexer, right_indexer = self._get_join_info() 583 584 ldata, rdata = self.left._data, self.right._data /data2/user/anaconda2/lib/python2.7/site-packages/pandas/core/reshape/merge.pyc in _get_join_info(self) 746 else: 747 (left_indexer, ---> 748 right_indexer) = self._get_join_indexers() 749 750 if self.right_index: /data2/user/anaconda2/lib/python2.7/site-packages/pandas/core/reshape/merge.pyc in _get_join_indexers(self) 725 self.right_join_keys, 726 sort=self.sort, ---> 727 how=self.how) 728 729 def _get_join_info(self): /data2/user/anaconda2/lib/python2.7/site-packages/pandas/core/reshape/merge.pyc in _get_join_indexers(left_keys, right_keys, sort, how, **kwargs) 1048 1049 # get left & right join labels and num. of levels at each location -> 1050 llab, rlab, shape = map(list, zip(* map(fkeys, left_keys, right_keys))) 1051 1052 # get flat i8 keys from label lists TypeError: type object argument after * must be an iterable, not itertools.imap
错误原因拆解
这个错误核心是Python 2.7与Pandas版本的兼容性冲突。你现在用的是Python 2.7,而这个版本早在2020年就停止维护了,后续的Pandas版本逐渐放弃了对它的支持。
具体来说,错误发生在Pandas内部的合并逻辑中:代码尝试用zip(* map(fkeys, left_keys, right_keys))来处理连接键,但在Python 2.7里,itertools.imap返回的是迭代器对象,而旧版Pandas的代码没有正确处理这种迭代器的解包操作,导致抛出“无法解包非可迭代对象”的类型错误。
解决方法(按优先级排序)
1. 升级到Python 3(强烈推荐)
Python 2.7已经被淘汰多年,包括Pandas在内的几乎所有主流库都不再为它提供更新和bug修复。升级到Python 3.8及以上版本,不仅能彻底解决这类兼容性问题,还能获得更好的性能、更多新特性,以及更活跃的社区支持。
2. 降级Pandas到兼容Python 2.7的稳定版本
如果暂时无法升级Python,你可以安装最后一批支持Python 2.7的Pandas版本(比如0.25.x系列)。执行以下命令即可:
pip install pandas==0.25.3
注意:安装前最好先备份当前的Python环境(比如用conda创建虚拟环境),避免破坏现有依赖。
3. 临时 workaround:统一连接键类型并转换为列表
如果不想改动环境,可以尝试先确保两个DataFrame的连接键类型完全一致,再执行合并:
# 显式将所有连接键转换为字符串类型(避免隐性类型差异) pio_smp2_sm['entityId'] = pio_smp2_sm['entityId'].astype(str) pio_smp2_sm['targetEntityId'] = pio_smp2_sm['targetEntityId'].astype(str) cm_smp2['user_id'] = cm_smp2['user_id'].astype(str) cm_smp2['product_id'] = cm_smp2['product_id'].astype(str) # 再执行合并 comp_df2 = pd.merge( pio_smp2_sm, cm_smp2, how='inner', left_on=['entityId','targetEntityId'], right_on=['user_id','product_id'] )
这个方法不一定能100%解决问题,但可以排除因连接键类型不一致导致的隐性问题,有时能绕过版本兼容的bug。
额外检查项
- 再次确认连接键没有缺失值:虽然
info()显示都是non-null,但可以用pio_smp2_sm[['entityId','targetEntityId']].isnull().sum()再次验证,避免合并时因隐藏缺失值导致的异常。 - 检查连接键的内容格式:比如是否存在前后空格、特殊字符等,这些也可能导致合并失败或异常。
内容的提问来源于stack exchange,提问作者user3476463

