Pandas双DataFrame嵌套迭代匹配unique_id时出现重复结果问题求助

Let's break down why you're getting duplicate names and how to fix it, plus a far better approach than nested loops (since those are slow and error-prone for Pandas).

The Root of Your Problem #

Your nested loops iterate over every row in dfA, then every row in dfB for each dfA row. Even after finding a matching unique_id, the inner loop keeps running—if dfB has multiple rows with the same unique_id, or dfA has duplicate unique_ids, you'll append the same name multiple times.

Quick Fix for Your Existing Loop #

If you want to stick with loops for now, add a break right after appending the name. This stops the inner loop as soon as it finds a match, preventing redundant checks and duplicates:

empty_list = [] 
for i, r in dfA.iterrows(): 
    for j, ro in dfB.iterrows(): 
        if r['unique_id'] == ro['unique_id']: 
            empty_list.append(ro['name']) 
            print(r['unique_id'], ro['unique_id'], ro['name']) 
            break  # Exit inner loop immediately after matching
        else: 
            pass

Note: This still won't fix duplicates if dfA has repeated unique_ids. If you only want one entry per unique ID, add dfA = dfA.drop_duplicates(subset='unique_id') before the loops.

Better: Use Pandas Built-In Functions (Faster & Cleaner) #

Nested loops are inefficient for Pandas—use vectorized operations instead. Here are two optimal approaches:

1. Merge DataFrames

Merge dfA and dfB on unique_id, then clean up duplicates and extract your list:

# Merge only the columns we need (unique_id from dfA, unique_id + name from dfB)
merged_df = dfA[['unique_id']].merge(dfB[['unique_id', 'name']], on='unique_id', how='left')

# Keep only one entry per unique_id
merged_df = merged_df.drop_duplicates(subset='unique_id')

# Convert to your target list
empty_list = merged_df['name'].tolist()

# Print in your desired format
for uid, name in zip(merged_df['unique_id'], merged_df['name']):
    print(f"{uid}. {name}")

2. Create a Lookup Map

If dfB has unique unique_ids, turn it into an index-based lookup for lightning-fast access:

# Create a Series where index = unique_id, value = name
name_lookup = dfB.set_index('unique_id')['name']

# Get unique IDs from dfA and map to names
unique_ids = dfA['unique_id'].unique()
empty_list = name_lookup.loc[unique_ids].tolist()

# Print results
for uid, name in zip(unique_ids, empty_list):
    print(f"{uid}. {name}")

Why These Are Better #

• Speed: Pandas vectorized operations are orders of magnitude faster than loops, especially with large datasets.
• Readability: The code clearly expresses what you're doing (matching IDs, getting names) instead of getting lost in loop logic.
• Less Error-Prone: No need to manually handle loop breaks or duplicate checks—Pandas does it for you.

内容的提问来源于stack exchange，提问作者Brad