如何在无Pandas依赖的Python CSV连接中移除指定字段？

Since you can't rely on Pandas due to server dependency restrictions, Python's standard csv module has everything you need to handle this join and field removal task. Here's a straightforward implementation:

Core Approach #

1. Load the first CSV and store its rows in a dictionary, keyed by your matching column (so we can quickly look up rows when processing the second CSV).
2. Load the second CSV, filter out the unwanted fields (like XID) from each row and its header.
3. For each row in the second CSV, find the matching row from the first CSV, combine them, and write the merged row to the output CSV.

Example Code #

import csv

# Configuration - adjust these to your specific needs
MATCH_KEY = "common_id"  # Replace with your actual matching column name
UNWANTED_FIELDS = {"XID"}  # Fields from Table 2 you want to remove
TABLE1_PATH = "table1.csv"
TABLE2_PATH = "table2.csv"
OUTPUT_PATH = "merged_output.csv"

# Step 1: Read Table 1 into a dictionary for fast lookups
table1_data = {}
with open(TABLE1_PATH, mode='r', newline='', encoding='utf-8') as f:
    reader = csv.DictReader(f)
    table1_fields = reader.fieldnames
    for row in reader:
        key = row[MATCH_KEY]
        table1_data[key] = row

# Step 2: Process Table 2, filter unwanted fields, and merge with Table 1
with open(TABLE2_PATH, mode='r', newline='', encoding='utf-8') as table2_f, \
     open(OUTPUT_PATH, mode='w', newline='', encoding='utf-8') as output_f:
    
    table2_reader = csv.DictReader(table2_f)
    # Filter out unwanted fields from Table 2's header
    filtered_table2_fields = [field for field in table2_reader.fieldnames if field not in UNWANTED_FIELDS]
    # Combine headers from both tables (skip duplicate match key)
    output_fields = table1_fields + [field for field in filtered_table2_fields if field != MATCH_KEY]
    
    writer = csv.DictWriter(output_f, fieldnames=output_fields)
    writer.writeheader()
    
    for table2_row in table2_reader:
        key = table2_row[MATCH_KEY]
        # This does an inner join (only rows with matches in both tables are kept)
        if key in table1_data:
            merged_row = table1_data[key].copy()
            # Add filtered Table 2 fields to the merged row
            for field in filtered_table2_fields:
                if field != MATCH_KEY:
                    merged_row[field] = table2_row[field]
            writer.writerow(merged_row)

print(f"Merged CSV saved to {OUTPUT_PATH}")

Key Adjustments & Notes #

• Join Type: The example uses an inner join. If you need a left join (keep all rows from Table 1 even if no match in Table 2), loop through Table 1 instead and merge with Table 2 rows if they exist.
• Unwanted Fields: Just add more field names to the UNWANTED_FIELDS set if you need to remove additional columns from Table 2.
• Encoding: The code uses utf-8—swap this for latin-1 or another encoding if your CSVs require it.
• Duplicate Keys: If Table 1 has duplicate matching keys, this code keeps the last occurrence. Modify the dictionary to store lists of rows if you need to handle duplicates.

This uses only Python's built-in tools, so no extra installations are needed—perfect for your server environment.

内容的提问来源于stack exchange，提问作者KevKosDev