pytest测试结果不匹配，CSV读写数据疑似损坏问题排查

问题背景 #

编写了用于处理CSV文件的CSVFile类，并编写了pytest参数化测试用例，运行测试后发现读取的数据与写入的数据不一致，报错显示读取到的是字符串形式的列表而非预期的列表元素。

CSVFile类代码 #

import csv
import os

class CSVFile:
    def __init__(self, filename):
        self.filename = filename

    def write_csv_line(self, row, mode):
        for value in row:
            if value == "None":
                value = ""
        writer = csv.writer(open(self.filename, mode))
        writer.writerow(row)

    def read_file(self):
        data = []
        with open(self.filename, mode="r") as csvfile:
            csv_data = csv.reader(csvfile, delimiter=",")
            for row in csv_data:
                data.append(row)
            return data

    def file_exists(self):
        return os.path.exists(self.filename)

    def delete(self):
        if os.path.exists(self.filename):
            os.remove(self.filename)
            return True
        else:
            return False

pytest测试代码 #

import pytest
from pytest_lazyfixture import lazy_fixture
from your_module import CSVFile  # 替换为实际模块名

@pytest.fixture(scope="function")
def csv_file():
    csv_file = CSVFile("test")
    yield csv_file
    csv_file.delete()

# Test if file is writable and correct data written
# @pytest.mark.skip("WIP")
@pytest.mark.parametrize(
    "file, row, expected",
    [
        (lazy_fixture("csv_file"), [["a", "b", "c"]], [["a", "b", "c"]]),
    ],
)
def test_CSVLineWritable(file, row, expected):
    file.write_csv_line(row, "w")
    data_read = file.read_file()
    assert file.file_exists() is True
    assert data_read == expected

报错信息 #

file = <process_resources.CSVFile object at 0x108a64af0>, row = [['a', 'b', 'c']], expected = [['a', 'b', 'c']]

    @pytest.mark.parametrize(
        "file, row, expected",
        [
            (lazy_fixture("csv_file"), [["a", "b", "c"]], [["a", "b", "c"]]),
            # (lazy_fixture("csv_file"), [["None", "b", "c"]], [["", "b", "c"]]),
            # (lazy_fixture("csv_file"), [[None, "b", "c"]], [["", "b", "c"]]),
        ],
    )
    def test_CSVLineWritable(file, row, expected):
        file.write_csv_line(row, "w")
        data_read = file.read_file()
        assert file.file_exists() is True
>       assert data_read == expected
E       assert [["['a', 'b', 'c']"]] == [['a', 'b', 'c']]
E         At index 0 diff: ["['a', 'b', 'c']"] != ['a', 'b', 'c']
E         Full diff:
E         - [['a', 'b', 'c']]
E         + [["['a', 'b', 'c']"]]
E         ?  ++               + +

tests/test_process_resources.py:117: AssertionError

问题原因及修复方案 #

1. 测试用例传入了嵌套列表 #

测试用例中传入的row参数是嵌套列表[["a", "b", "c"]]，而csv.writer.writerow()方法接收的是一维列表（代表一行的所有元素）。当传入嵌套列表时，writerow会把内部的列表["a","b","c"]直接转换成字符串"['a', 'b', 'c']"写入CSV文件，因此读取后得到的就是包含该字符串的列表[["['a', 'b', 'c']"]]，与预期不符。

修复测试用例：将row改为一维列表，对应的expected保持二维列表（因为read_file返回的是所有行组成的二维列表）：

@pytest.mark.parametrize(
    "file, row, expected",
    [
        (lazy_fixture("csv_file"), ["a", "b", "c"], [["a", "b", "c"]]),
    ],
)

2. write_csv_line方法的循环未实际修改元素 #

原方法中的循环：

for value in row:
    if value == "None":
        value = ""

只是修改了临时变量value，并没有改变原列表row中的元素，导致遇到值为"None"的元素时无法替换为空字符串。

修复方法：使用列表推导式或索引遍历修改原列表：

# 方式1：列表推导式（推荐）
def write_csv_line(self, row, mode):
    # 替换"None"为空字符串
    processed_row = ["" if value == "None" else value for value in row]
    with open(self.filename, mode, newline='') as f:
        writer = csv.writer(f)
        writer.writerow(processed_row)

# 方式2：索引遍历修改原列表
def write_csv_line(self, row, mode):
    for i in range(len(row)):
        if row[i] == "None":
            row[i] = ""
    with open(self.filename, mode, newline='') as f:
        writer = csv.writer(f)
        writer.writerow(row)

3. 文件未正确关闭（额外优化） #

原方法中直接使用open(self.filename, mode)而未用with语句，可能导致文件未及时刷新关闭，引发读取异常。使用with语句可以确保文件自动关闭，避免资源泄漏和数据写入不完整的问题。

内容的提问来源于stack exchange，提问作者Burvil