如何检测DataFrame中数值列之间的数值关系？

我从数据库中提取了数值列的子集，想要遍历这些列，找到满足target_column等于另外两列数值运算结果（比如col1 * col2 = target_column）的关系，但不确定怎么高效实现。我写了一段代码，想知道这是不是最优的实现方式：

# For all possible combinations of numeric columns
for col1, col2 in combinations(numeric_cols, 2):
    # For a target column in numeric_columns
    for target_column in numeric_cols:
        # Skip if the target column is one of the relationship columns
        if target_column in (col1, col2):
            continue

def analyse_relationships(df):

numeric_cols = df.select_dtypes(include=[np.number])
threshold = 0.001
relationships = []

# For all possible combinations of numeric columns
for col1, col2 in combinations(numeric_cols, 2):
    # For a target column in numeric_columns
    for target_column in numeric_cols:
        # Skip if the target column is one of the relationship columns
        if target_column in (col1, col2):
            continue

        # Calculate different operations
        product = numeric_cols[col1] * numeric_cols[col2]
        sum_cols = numeric_cols[col1] + numeric_cols[col2]
        diff = numeric_cols[col1] - numeric_cols[col2]

        if np.allclose(product, numeric_cols[target_column], rtol=threshold):
            relationships.append(f"{col1} * {col2} = {target_column}")
        elif np.allclose(sum_cols, numeric_cols[target_column], rtol=threshold):
            relationships.append(f"{col1} + {col2} = {target_column}")
        elif np.allclose(diff, numeric_cols[target_column], rtol=threshold):
            relationships.append(f"{col1} - {col2} = {target_column}")

你的核心思路没问题，但可以从减少重复计算、缩小循环范围两个维度优化效率，同时提升代码可读性：

优化后的代码 #

import numpy as np
import pandas as pd
from itertools import combinations

def analyse_relationships(df):
    # 提取数值列名列表（避免后续操作中处理DataFrame类型的列集合）
    numeric_cols = df.select_dtypes(include=[np.number]).columns.tolist()
    threshold = 0.001
    relationships = []

    # 遍历所有两列组合
    for col1, col2 in combinations(numeric_cols, 2):
        # 提前获取列的数值数组，避免重复索引DataFrame
        col1_vals = df[col1].values
        col2_vals = df[col2].values
        
        # 一次计算三种运算结果，供所有target列复用
        product = col1_vals * col2_vals
        sum_cols = col1_vals + col2_vals
        diff_col1_col2 = col1_vals - col2_vals
        diff_col2_col1 = col2_vals - col1_vals

        # 预过滤target候选列：直接排除col1和col2，减少循环次数
        target_candidates = [col for col in numeric_cols if col not in (col1, col2)]
        for target_col in target_candidates:
            target_vals = df[target_col].values
            # 依次检查运算关系
            if np.allclose(product, target_vals, rtol=threshold):
                relationships.append(f"{col1} * {col2} = {target_col}")
            elif np.allclose(sum_cols, target_vals, rtol=threshold):
                relationships.append(f"{col1} + {col2} = {target_col}")
            elif np.allclose(diff_col1_col2, target_vals, rtol=threshold):
                relationships.append(f"{col1} - {col2} = {target_col}")
            elif np.allclose(diff_col2_col1, target_vals, rtol=threshold):
                relationships.append(f"{col2} - {col1} = {target_col}")
    return relationships

关键优化点 #

1. 提前缓存运算结果：原代码中每个target列都会重新计算一次乘、加、减，优化后对每组col1+col2只计算一次运算结果，大幅减少重复计算量。
2. 预过滤target列：直接生成排除col1/col2的候选列表，避免循环内的if分支判断，减少不必要的迭代。
3. 补充反向减法：原代码仅检查col1-col2，补充col2-col1后覆盖更多可能的数值关系。
4. 用NumPy数组运算：直接提取列的.values数组进行运算，比反复索引DataFrame更高效。

大数据集额外优化建议 #

如果处理的数据集规模很大，还可以尝试：

• 先做列相关性分析，优先检查相关性高的列组合，减少无意义运算；
• 用numba对数值运算部分做JIT编译，进一步提升运算速度；
• 分块处理数据，避免内存占用过高。

内容的提问来源于stack exchange，提问作者Evan Lynch