同时按索引与名称提取DataFrame列时遇警告及空列问题求助

Hey there! Let's break down why you're hitting that FutureWarning and getting empty columns, then fix it up properly.

What's Causing the Issue? #

When you run train_data.iloc[:, DATA_TYPE].loc[:, FEATURES], here's the root of the problem:

1. First, iloc[:, DATA_TYPE] extracts columns where the DATA_TYPE list has True at the corresponding index.
2. Then, .loc[:, FEATURES] tries to pull columns with names in your FEATURES list from that filtered DataFrame.

The warning and empty columns happen because some names in FEATURES don't exist in the columns you extracted in the first step. Pandas currently creates empty columns for these missing names, but the FutureWarning is a heads-up: in future versions, this will throw a KeyError instead (a stricter behavior that avoids accidental empty columns).

Solution 1: One-Step Selection (Cleanest Approach) #

Instead of splitting the selection into two steps, we can directly find columns that meet both criteria (DATA_TYPE is True AND column name is in FEATURES) in one go:

# Get all column names from the original training data
all_columns = train_data.columns.tolist()

# Filter columns that match both conditions
selected_columns = [
    col for idx, col in enumerate(all_columns)
    if DATA_TYPE[idx] and col in FEATURES
]

# Extract the valid columns directly
result = train_data.loc[:, selected_columns]

Solution 2: Fix the Two-Step Workflow #

If you prefer to keep the two-step structure, just filter FEATURES to only include names that exist in the intermediate DataFrame:

# Step 1: Extract columns where DATA_TYPE is True
temp_data = train_data.iloc[:, DATA_TYPE]

# Step 2: Keep only FEATURES that are present in temp_data's columns
valid_features = [col for col in FEATURES if col in temp_data.columns]

# Extract the valid columns
result = temp_data.loc[:, valid_features]

Why This Works #

Both approaches ensure you only request columns that actually exist in the DataFrame at the time of selection. This eliminates empty columns, silences the FutureWarning, and makes your code compatible with future pandas versions.

内容的提问来源于stack exchange，提问作者Boyang Li