Pandas IndexingError排查：布尔序列与索引不匹配问题

需求 #

仅保留数据集kirp_deg_gene_exp中subtype行值为KIRP或normal的列，执行以下代码时触发错误：

df_kirp = df_kirp.sort_index()

mrna_clin = mrna_clin.sort_index()
mrna_clin = mrna_clin.iloc[:,7:]

# Keep only columns of df that has clin_type info
kirp_deg_gene_exp = mrna_clin[mrna_clin.columns.intersection(df_kirp.index)].T
kirp_deg_gene_exp = pd.concat([kirp_deg_gene_exp, subtype])
kirp_deg_gene_exp = kirp_deg_gene_exp[kirp_deg_gene_exp.loc['subtype'].str.contains("KIRP|normal")]

报错回溯 #

/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py:4: UserWarning: Boolean Series key will be reindexed to match DataFrame index.
  after removing the cwd from sys.path.
---------------------------------------------------------------------------
IndexingError                             Traceback (most recent call last)
/tmp/ipykernel_27/274904861.py in <module>
      2 kirp_deg_gene_exp = mrna_clin[mrna_clin.columns.intersection(df_kirp.index)].T
      3 kirp_deg_gene_exp = pd.concat([kirp_deg_gene_exp, subtype])
----> 4 kirp_deg_gene_exp = kirp_deg_gene_exp[kirp_deg_gene_exp.loc['subtype'].str.contains("KIRP|normal")]
      5 kirp_deg_gene_exp

/opt/conda/lib/python3.7/site-packages/pandas/core/frame.py in __getitem__(self, key)
   3447         # Do we have a (boolean) 1d indexer?
   3448         if com.is_bool_indexer(key):
-> 3449             return self._getitem_bool_array(key)
   3450 
   3451         # We are left with two options: a single key, and a collection of keys,

/opt/conda/lib/python3.7/site-packages/pandas/core/frame.py in _getitem_bool_array(self, key)
   3500         # check_bool_indexer will throw exception if Series key cannot
   3501         # be reindexed to match DataFrame rows
-> 3502         key = check_bool_indexer(self.index, key)
   3503         indexer = key.nonzero()[0]
   3504         return self._take_with_is_copy(indexer, axis=0)

/opt/conda/lib/python3.7/site-packages/pandas/core/indexing.py in check_bool_indexer(index, key)
   2387         if mask.any():
   2388             raise IndexingError(
-> 2389                 "Unalignable boolean Series provided as "
   2390                 "indexer (index of the boolean Series and of "
   2391                 "the indexed object do not match)."

IndexingError: Unalignable boolean Series provided as indexer (index of the boolean Series and of the indexed object do not match).

错误原因 #

你生成的布尔序列是基于subtype行的列索引，但直接用该序列索引DataFrame时，pandas默认会尝试匹配DataFrame的行索引，两者索引维度不匹配，因此触发IndexingError。本质是你要筛选的是列，却用了默认的行索引筛选逻辑。

修复方案 #

方案一：明确指定筛选列（推荐） #

修改最后一行代码，用loc[:, 布尔序列]明确告诉pandas要操作列轴：

kirp_deg_gene_exp = kirp_deg_gene_exp.loc[:, kirp_deg_gene_exp.loc['subtype'].str.contains("KIRP|normal")]

方案二：先对齐索引再筛选 #

如果subtype是独立的Series，先确保其索引与kirp_deg_gene_exp的列索引对齐，再生成筛选掩码：

# 对齐subtype索引与目标DataFrame的列索引
subtype_aligned = subtype.reindex(kirp_deg_gene_exp.columns)
# 生成布尔筛选掩码
mask = subtype_aligned.str.contains("KIRP|normal")
# 筛选符合条件的列
kirp_deg_gene_exp = kirp_deg_gene_exp.loc[:, mask]

内容的提问来源于stack exchange，提问作者melolilili