Pandas中无法删除含特定缺失值行的问题排查

Hey there, let's break down exactly why your initial code wasn't removing the rows you expected, and how to fix it.

The Core Logic Mistake #

Your original code uses logical OR (|) when you actually need logical AND (&). Let's unpack this:

Your first filter line:

a = a.loc[(a['A1_TOP'] != '-') | (a['A2_TOP'] != '-')]

This tells Pandas: "Keep any row where A1_TOP isn't '-' OR A2_TOP isn't '-'."

Take your example row:

Name Sample A1_TOP A2_TOP
Adam Smith - B

Here, A2_TOP isn't '-', so the OR condition evaluates to True—meaning Pandas keeps this row, even though A1_TOP is '-'. That's why it wasn't getting deleted.

Why Your Split Code Works #

When you split the filters into two separate lines:

a = a.loc[a['A1_TOP'] != '-']
a = a.loc[a['A2_TOP'] != '-']

This is equivalent to saying: "First keep only rows where A1_TOP isn't '-', then from those results, keep only rows where A2_TOP isn't '-'."

In logical terms, this is a logical AND operation—both conditions need to be true for a row to stay. That's why this version correctly removes rows where either column has '-'.

Fixing Your Original Code #

To make your original approach work, swap the | operators for & (and combine the checks for '-' and '0' properly):

# Keep rows where neither column is '-' OR '0'
a = a.loc[(a['A1_TOP'] != '-') & (a['A2_TOP'] != '-') & 
          (a['A1_TOP'] != '0') & (a['A2_TOP'] != '0')]

A Cleaner Alternative #

For better readability, you can use isin() to define your missing values in one place:

missing_vals = ['-', '0']
# ~ means "not", so we keep rows where neither column is in missing_vals
a = a.loc[~a['A1_TOP'].isin(missing_vals) & ~a['A2_TOP'].isin(missing_vals)]

Quick Recap #

• OR (|): Keeps rows where at least one condition is true (your original code kept rows with only one missing value)
• AND (&): Keeps rows where all conditions are true (what you actually needed to remove any row with a missing value)
• Split filters act like AND because you're narrowing down the DataFrame step by step

内容的提问来源于stack exchange，提问作者martin