sklearn多进程设置不同导致PCA结果不一致问题求助

在使用sklearn的multiprocessing模块时，遇到了PCA相关的异常问题：PCA的reconstruction error会随Pool设置的进程数变化出现显著差异。例如设置Pool(processes=4)时误差极小（np.abs(tmp_matrix-X_train).max()<1e-2），但进程数增至5及以上时，np.abs(tmp_matrix-X_train).max()的平均值约为10。该问题仅出现在使用Intel sklearnex包的场景中。

测试观察到的规律 #

• 稳定（低误差）场景：20 cpu+processes=1、80 cpu+processes=1、80 cpu+processes=4、120 cpu+processes=5
• 不稳定（高误差）场景：80 cpu+processes=5、100 cpu+processes=5、120 cpu+processes=5（注：120核+5进程的情况也会出现不稳定）

相关代码片段 #

from sklearnex import patch_sklearn
patch_sklearn()
from sklearn.decomposition import PCA
from functools import partial
from multiprocessing import Pool

def config_selection_single(df_entry: tuple, _some_arguments_indlucding_data_object):
    #some pre-processing code
    for some_iteration_condition:
        # some data processing and transformation to bound data non-NaN and between [-1e20,1e20]
        for another_iteration_condition:
            z_mean = X[train_cond][:].mean()    
            z_std = X[train_cond][:].std()+1e-10
            X_train = (X[train_cond][:]-z_mean) / z_std # X_train has shape ~ 2e4 X 50

            pca = PCA(n_components=20, svd_solver='full')
            p_model = pca.fit(X_train)
            Q = p_model.transform(X_train)
            tmp_matrix = p_model.inverse_transform(Q)
            if not np.allclose(Q,X_train.dot(p_model.components_.transpose())): # to compute reconstruction error.
                print("reconstruction error is huge!")
                print(np.abs(tmp_matrix-X_train).max())
        
config_selection_prtial = partial(config_selection_single, _some_arguments_indlucding_data_object)
with Pool(processes=4) as pool: # 4 is good, 5 and 6 are bad
    pool.map(config_selection_prtial, list(my_df.items()))

遗憾的是未能找到可复现该问题的小型数据集。

请问为何进程数会影响PCA的精度？

内容的提问来源于stack exchange，提问作者cat