如何读取含不规则引号的CSV字符串为pandas DataFrame？

问题原因 #

你遇到的问题是CSV行采用了双引号转义写法：整行被双引号包裹，内部包含逗号的数值用""...""（两个双引号）来表示带逗号的数值，这导致pandas默认解析规则无法正确识别分隔符。

方法1：直接调整read_csv参数 #

只需要修改pd.read_csv的几个参数，就能让pandas正确识别这种格式：

import io
import pandas as pd

text="""A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P
"2019,01/2019,Brazil,0,""166,229"",0,""22578,86292"",""47,417"",,,,,,,,"
"2019,01/2019,Brazil,95,0,""50,34563"",0,""5,137"",,,,,,,,"
2019,01/2019,Brazil,0,0,0,0,0,,,,,,,,"""

df = pd.read_csv(
    io.StringIO(text),
    sep=',',
    engine="python",
    encoding='utf-8',
    decimal=",",
    quotechar='"',  # 指定引号字符为双引号
    doublequote=True,  # 启用双引号转义（即""表示一个"）
    skipinitialspace=True  # 可选，避免分隔后出现多余空格
)

print(df)

运行后所有列会被正确拆分，带逗号的数值也会按decimal=","的规则转为浮点数。

方法2：预处理文本（参数调整无效时用） #

如果第一种方法不生效，可以先手动修正CSV格式，再解析：

import io
import pandas as pd

text="""A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P
"2019,01/2019,Brazil,0,""166,229"",0,""22578,86292"",""47,417"",,,,,,,,"
"2019,01/2019,Brazil,95,0,""50,34563"",0,""5,137"",,,,,,,,"
2019,01/2019,Brazil,0,0,0,0,0,,,,,,,,"""

# 预处理步骤：替换""为"，去掉每行首尾的双引号
processed_lines = []
for line in text.splitlines():
    cleaned_line = line.replace('""', '"')
    if cleaned_line.startswith('"') and cleaned_line.endswith('"'):
        cleaned_line = cleaned_line[1:-1]
    processed_lines.append(cleaned_line)

processed_text = "\n".join(processed_lines)

# 正常解析处理后的文本
df = pd.read_csv(
    io.StringIO(processed_text),
    sep=',',
    engine="python",
    encoding='utf-8',
    decimal=","
)

print(df)

这个方法通过手动修正格式，让pandas能按常规规则解析。

内容的提问来源于stack exchange，提问作者user026