Python中不等行数DataFrame分组计算spearmanr的随机抽样方案

直接封装一个通用计算函数，自动完成样本长度对齐再计算相关系数即可，核心逻辑是先剔除两个序列的空值，对比有效样本长度，对更长的序列做无放回随机抽样，匹配到短序列的样本量后再传入spearmanr计算。

完整代码 #

import pandas as pd
from scipy import stats

def calc_aligned_spearman(s1, s2, random_seed=42, **kwargs):
    # 先剔除空值，统计有效样本量
    s1_valid = s1.dropna()
    s2_valid = s2.dropna()
    len_s1, len_s2 = len(s1_valid), len(s2_valid)
    
    # 样本量一致直接计算
    if len_s1 == len_s2:
        return stats.spearmanr(s1_valid, s2_valid, **kwargs)
    
    # 取最短样本量为对齐基准，长序列无放回随机抽样
    min_sample = min(len_s1, len_s2)
    s1_aligned = s1_valid.sample(n=min_sample, random_state=random_seed, replace=False) if len_s1 > min_sample else s1_valid
    s2_aligned = s2_valid.sample(n=min_sample, random_state=random_seed, replace=False) if len_s2 > min_sample else s2_valid
    
    return stats.spearmanr(s1_aligned, s2_aligned, **kwargs)

# 以下为原有逻辑，替换计算部分即可
df = pd.read_csv('Dafniyot_Data.csv', delimiter=',')
# 注意：如果group字段是精确的三位分组值，建议把str.contains换成==等值匹配，避免子串误匹配
# 例：CII = df[df['group'] == 'CII']
CII = df[df['group'].str.contains('CII')]
CCI = df[df['group'].str.contains('CCI')]
CCC = df[df['group'].str.contains('CCC')]
III = df[df['group'].str.contains('III')]
CIC = df[df['group'].str.contains('CIC')]
ICC = df[df['group'].str.contains('ICC')]
IIC = df[df['group'].str.contains('IIC')]
ICI = df[df['group'].str.contains('ICI')]

# 调用封装函数计算，自动对齐样本
F1a = calc_aligned_spearman(CCC['N_offspring'], CIC['N_offspring'], nan_policy='omit')
F1b = calc_aligned_spearman(CCI['N_offspring'], CII['N_offspring'], nan_policy='omit')

说明 #

• 函数内置固定随机种子random_seed=42，保证每次运行抽样结果一致，结果可复现，需要更换随机抽样结果修改该参数值即可
• 抽样采用无放回模式，避免重复抽取同一样本干扰计算结果
• 计算前预先剔除空值，避免空值占用样本名额导致有效样本量对齐错误
• 如果你分组用的str.contains是模糊匹配，注意排查是否存在子串重叠误分组的情况，比如字符串CCII会同时匹配CCI和CII的规则，如果group字段本身就是固定三位的分组编码，直接用等值判断==更稳妥

内容的提问来源于stack exchange，提问作者Rimidelo