如何在Pandas中获取行内相等或近似值并筛选符合条件的行？

Got it, let's tackle these two Pandas tasks step by step. I'll use your sample data (I filled in the truncated 9th row for completeness) to demonstrate everything clearly.

First, let's set up the sample DataFrame:

import pandas as pd

data = {
    'a': [1128.70, 1130.20, 1113.40, 1126.25, 1124.45, 1137.90, 1097.60, 1111.05, 1092.75, 1083.60],
    'b': [1137.00, 1142.30, 1127.90, 1129.30, 1141.10, 1141.90, 1117.00, 1119.10, 1097.60, 1091.20],
    'c': [1121.30, 1109.10, 1109.85, 1111.20, 1121.00, 1094.50, 1095.65, 1089.85, 1074.10, 1070.45],
    'd': [1132.05, 1114.90, 1124.55, 1124.50, 1137.95, 1098.25, 1112.50, 1092.10, 1083.75, 1078.10]
}
df = pd.DataFrame(data)

1. 获取某一行中相等或近似相等的数值 #

We can write a custom function that takes a target row and a threshold (like 0.5 or 1) to find all value pairs in the row that meet the similarity requirement:

def find_close_values(row, threshold=0.5):
    values = row.values
    cols = row.index
    close_pairs = []
    
    # Check all unique column pairs to avoid duplicates
    for i in range(len(cols)):
        for j in range(i + 1, len(cols)):
            diff = abs(values[i] - values[j])
            if diff <= threshold:
                close_pairs.append({
                    'columns': (cols[i], cols[j]),
                    'values': (values[i], values[j]),
                    'difference': round(diff, 2)
                })
    return close_pairs

# Example: Find values with ≤0.5 difference in row index 4
row_4_results = find_close_values(df.loc[4], threshold=0.5)
print("Close value pairs in row 4:")
for pair in row_4_results:
    print(f"Columns {pair['columns']}: Values {pair['values']}, Difference {pair['difference']}")

You can tweak the threshold parameter to match your needs—crank it up to 3 if you want to include pairs with a larger gap, or set it to 0 to find exactly equal values.

2. 筛选满足条件的行 #

We need to filter rows that meet any of these criteria:

• At least 2 columns have equal or nearly equal values (within your threshold)
• Multiple columns (≥2) meet the similarity requirement

First, write a helper function to check if a row qualifies, then use df.apply() to filter the DataFrame:

def row_meets_criteria(row, threshold=1.0):
    values = row.values
    valid_pairs = 0
    
    # Count how many column pairs meet the threshold
    for i in range(len(values)):
        for j in range(i + 1, len(values)):
            if abs(values[i] - values[j]) <= threshold:
                valid_pairs += 1
    
    # Return True if at least one valid pair exists (i.e., ≥2 columns are similar)
    return valid_pairs >= 1

# Filter rows with a threshold of 1.0
filtered_df = df[df.apply(row_meets_criteria, threshold=1.0, axis=1)]
print("Filtered rows:")
print(filtered_df)

Pro Tip for Large Datasets #

The double-loop method works fine for small tables, but for large datasets, use numpy's vectorized operations to speed things up:

import numpy as np

def row_meets_criteria_fast(row, threshold=1.0):
    arr = row.values.reshape(-1, 1)
    # Calculate absolute differences between all value pairs
    diff_matrix = np.abs(arr - arr.T)
    # Check if any off-diagonal pairs meet the threshold (exclude self-comparisons)
    return np.any(diff_matrix[np.triu_indices_from(diff_matrix, k=1)] <= threshold)

# Faster filtering for big data
fast_filtered_df = df[df.apply(row_meets_criteria_fast, threshold=1.0, axis=1)]

If you want to only keep rows with exactly equal values, just set threshold=0 in either function.

内容的提问来源于stack exchange，提问作者Don Coder