Python导入CSV后筛选多列值一致行及三列值相同数据过滤方法咨询

Hey there, let's break down how to solve both of your problems—first, the general case of filtering rows where three columns have identical values, then specifically for your four Performance test columns. We'll use pandas since it's the go-to tool for this kind of CSV data manipulation.

Sample Data #

First, let's clarify the sample data you provided (formatted as proper CSV):

ID,Name,Performance test 1,Performance test 2,Performance test 3,Performance test 4,Consistent?
1,Bob,Pass,Pass,Pass,Pass,TRUE
2,Dave,Pass,Fail,Pass,Pass,FALSE
3,Roger,Fail,Fail,Fail,Fail,TRUE

Solution 1: General Case (Filter Rows Where N Columns Are Identical) #

If you need to filter rows where any set of columns (like three columns) have the same values, the core idea is to check if all values in those columns for a row are equal.

For example, if you had columns ColA, ColB, ColC, you'd do this:

import pandas as pd

# Load your CSV file
df = pd.read_csv('your_data.csv')

# Define the columns you want to verify
target_columns = ['ColA', 'ColB', 'ColC']

# Filter rows where all values in target_columns are identical
filtered_df = df[df[target_columns].apply(lambda row: row.nunique() == 1, axis=1)]

• row.nunique() ==1 checks if there’s only one unique value in the row across the target columns (meaning all values match).
• axis=1 tells pandas to run the check row-by-row instead of column-by-column.

Solution 2: Specific to Your Performance Test Columns #

Now, applying this to your exact requirement—filtering rows where all four Performance test columns have the same value:

import pandas as pd

# Load the performance data CSV
df = pd.read_csv('performance_data.csv')

# Define the four test columns we care about
test_columns = [
    'Performance test 1',
    'Performance test 2',
    'Performance test 3',
    'Performance test 4'
]

# Filter rows where all test columns have identical values
consistent_rows = df[df[test_columns].apply(lambda row: row.nunique() == 1, axis=1)]

# Optional: Update or add the "Consistent?" column to match the sample
df['Consistent?'] = df[test_columns].apply(lambda row: row.nunique() ==1, axis=1).astype(str).upper()

Output of the Filtered Rows #

Running the code above on your sample data will return this consistent_rows DataFrame:

Alternative Method (Faster for Large Datasets) #

If you’re working with a big dataset, this method is slightly more efficient—it compares every column in the test set to the first one, then ensures all comparisons are true:

consistent_rows = df[df[test_columns].eq(df[test_columns].iloc[:,0], axis=0).all(axis=1)]

This checks if every value in the row matches the first value of the test columns for that row, which avoids calculating unique values for each row.

内容的提问来源于stack exchange，提问作者Nicola Hodge