Python CSV文件读写操作及客户搜索函数重复结果问题排查

1. 使用内置csv模块 #

Python标准库自带工具，无需额外安装，适合基础场景。

读取CSV文件

• 按行读取（列表格式）：

import csv

with open('data.csv', 'r', encoding='utf-8') as f:
    reader = csv.reader(f)
    headers = next(reader)  # 获取表头行
    for row in reader:
        print(row)  # 每行返回列表，如['张三', '138xxxxxxx', '2024-05-01']

• 按列名读取（字典格式）：

import csv

with open('data.csv', 'r', encoding='utf-8') as f:
    reader = csv.DictReader(f)
    for row in reader:
        print(row['customer_name'], row['order_date'])  # 直接通过列名取值

操作CSV文件（写入/修改）

• 写入新文件：

import csv

headers = ['customer_id', 'customer_name', 'phone']
rows = [
    ['001', '张三', '138xxxxxxx'],
    ['002', '李四', '139xxxxxxx']
]

with open('new_data.csv', 'w', encoding='utf-8', newline='') as f:
    writer = csv.writer(f)
    writer.writerow(headers)  # 写入表头
    writer.writerows(rows)  # 批量写入数据行

• 修改现有文件：先读取全部数据，修改后重新写入

import csv

# 读取原始数据
data = []
with open('data.csv', 'r', encoding='utf-8') as f:
    reader = csv.DictReader(f)
    data = list(reader)
    headers = reader.fieldnames

# 修改指定数据：更新张三的手机号
for row in data:
    if row['customer_name'] == '张三':
        row['phone'] = '137xxxxxxx'

# 写入修改后的数据
with open('data.csv', 'w', encoding='utf-8', newline='') as f:
    writer = csv.DictWriter(f, fieldnames=headers)
    writer.writeheader()
    writer.writerows(data)

2. 使用pandas库 #

适合处理大型CSV文件或复杂数据操作，需先安装：pip install pandas

读取和查看数据

import pandas as pd

df = pd.read_csv('data.csv')
print(df.head())  # 查看前5行数据
print(df.columns)  # 查看所有列名

常见操作

• 筛选数据：

# 筛选客户姓名包含"张"的记录
filtered_df = df[df['customer_name'].str.contains('张')]

• 修改数据：

# 更新张三的手机号
df.loc[df['customer_name'] == '张三', 'phone'] = '137xxxxxxx'

• 保存数据：

df.to_csv('updated_data.csv', index=False, encoding='utf-8')

常见原因及解决方法 #

1. 原始CSV数据存在重复条目

先检查订单CSV文件，若同一客户有多条重复订单记录，先对数据去重：

用csv模块去重（基于客户唯一标识，如customer_id）：

import csv

unique_data = {}
with open('orders.csv', 'r', encoding='utf-8') as f:
    reader = csv.DictReader(f)
    for row in reader:
        unique_data[row['customer_id']] = row

# 写入去重后的数据
with open('unique_orders.csv', 'w', encoding='utf-8', newline='') as f:
    writer = csv.DictWriter(f, fieldnames=reader.fieldnames)
    writer.writeheader()
    writer.writerows(unique_data.values())

用pandas去重更简便：

import pandas as pd

df = pd.read_csv('orders.csv')
df = df.drop_duplicates(subset=['customer_id'], keep='first')  # 保留第一条重复记录
df.to_csv('unique_orders.csv', index=False)

2. 搜索函数逻辑导致重复添加

若原始数据无重复，但搜索结果重复，大概率是遍历订单时，同一客户的多条订单都被加入结果列表。错误示例：

# 错误：同一客户有多条订单时，返回多条重复结果
def search_customers(keyword):
    results = []
    with open('orders.csv', 'r', encoding='utf-8') as f:
        reader = csv.DictReader(f)
        for row in reader:
            if keyword.lower() in row['customer_name'].lower():
                results.append(row)
    return results

修复：用字典存储结果，以客户唯一标识为键自动去重

def search_customers(keyword):
    results = {}  # 键为客户唯一标识，避免重复
    with open('orders.csv', 'r', encoding='utf-8') as f:
        reader = csv.DictReader(f)
        for row in reader:
            if keyword.lower() in row['customer_name'].lower():
                # 用customer_id作为唯一键，确保同一客户只存一次
                results[row['customer_id']] = row
    # 将字典值转为列表返回
    return list(results.values())

若无customer_id，可组合多字段作为唯一键：

key = f"{row['customer_name']}_{row['phone']}"
results[key] = row

3. 函数调用时的重复追加

若多次调用函数时结果被重复追加，检查是否在函数外部定义了结果列表，导致每次调用都往同一列表追加数据。确保结果列表在函数内部定义，每次调用生成全新列表。

内容的提问来源于stack exchange，提问作者tanaka tanaka