如何将模型训练结果的字符串列表转换为Pandas DataFrame？

我有多个存储模型训练/验证结果的.csv文件，每个文件对应一个模型，每行包含Epoch、loss_train、acc_train、loss_val、acc_val、time信息，文件名包含模型参数。我需要构建一个包含每个文件最后一行数据及对应文件名的DataFrame。

通过Python的readlines()方法，我已获取到包含所需信息的字符串列表，示例如下：

['"0_0_2_200_0.4.csv",66,67,0.42319968342781067,0.8733666720438781,0.9848468899726868,0.7532656023222061,0.2503340244293213\n', 
 '"0_0_2_200_0.5.csv",74,75,0.41233333945274353,0.8760283916760768,0.9206098318099976,0.7656023222060958,0.2535388469696045\n']

尝试以下方法均出现问题：

• 使用from_records方法：

  df = pd.DataFrame.from_records(results, columns = ["filename", "row_number", "Epoch", "loss_train", "acc_train","loss_val", "acc_val", "time"])
  

  报错：ValueError: 8 columns passed, passed data had 124 columns
• 直接创建DataFrame：

  df2 = pd.DataFrame(results,
                     columns=["filename", "row_number", "Epoch", "loss_train", "acc_train","loss_val", "acc_val", "time"])
  

  报错：ValueError: Shape of passed values is (110, 1), indices imply (110, 8)
• 直接转换得到仅含一列的DataFrame：

  df3 = pd.DataFrame(results)
  

• 将列表写入csv文件：

  file = open('final_results.csv', 'w+', newline ='') 
  with file:     
      write = csv.writer(file, delimiter=',') 
      write.writerows(results)
  

  生成的文件中所有字符被拆分到不同列，示例如下：

  """",0,_,0,_,2,_,2,0,0,_,0,.,4,.,c,s,v,"""",",6,6,",6,7,",0,.,4,2,3,1,9,9,6,8,3,4,2,7,8,1,0,6,7,",0,.,8,7,3,3,6,6,6,7,2,0,4,3,8,7,8,1,",0,.,9,8,4,8,4,6,8,8,9,9,7,2,6,8,6,8,",0,.,7,5,3,2,6,5,6,0,2,3,2,2,2,0,6,1,",0,.,2,5,0,3,3,4,0,2,4,4,2,9,3,2,1,3,"
  

请问如何将该字符串列表正确转换为DataFrame？

问题核心在于当前的字符串列表是完整的CSV行字符串，但未拆分成对应字段，无法直接传入DataFrame构造方法。以下是几种可行的解决方式：

方法1：手动拆分字符串并转换数据类型 #

遍历每个字符串，先清理格式再按逗号分割字段，最后转换数值类型：

import pandas as pd

# 假设results是你的目标字符串列表
processed_data = []
for line in results:
    # 去除换行符，去掉文件名前后的引号
    cleaned_line = line.strip().replace('"', '')
    # 按逗号分割成独立字段
    parts = cleaned_line.split(',')
    # 将数值字段转换为对应类型：整数或浮点数
    parts[1:] = [float(x) if '.' in x else int(x) for x in parts[1:]]
    processed_data.append(parts)

# 创建DataFrame并指定列名
df = pd.DataFrame(processed_data, columns=["filename", "row_number", "Epoch", "loss_train", "acc_train","loss_val", "acc_val", "time"])

方法2：利用StringIO让pandas直接读取CSV格式内容 #

把所有行字符串拼接成完整的CSV内容，用StringIO模拟文件对象，让pandas自动处理解析和类型转换：

import pandas as pd
from io import StringIO

# 拼接所有行，生成完整的CSV格式内容
csv_content = ''.join(results)
# 用StringIO模拟文件，pandas直接读取并指定列名
df = pd.read_csv(StringIO(csv_content), 
                 names=["filename", "row_number", "Epoch", "loss_train", "acc_train","loss_val", "acc_val", "time"])

方法3：修复csv.writer的写入逻辑 #

如果需要先写入CSV文件再读取，需先拆分每个字符串为字段列表，再写入：

import csv
import pandas as pd

file = open('final_results.csv', 'w+', newline='') 
with file:     
    write = csv.writer(file, delimiter=',') 
    # 先写入表头
    write.writerow(["filename", "row_number", "Epoch", "loss_train", "acc_train","loss_val", "acc_val", "time"])
    # 遍历处理每一行数据
    for line in results:
        cleaned_line = line.strip().replace('"', '')
        parts = cleaned_line.split(',')
        write.writerow(parts)

# 读取生成的CSV文件到DataFrame
df = pd.read_csv('final_results.csv')

内容的提问来源于stack exchange，提问作者imaspol