如何在Python中将稀疏DataFrame转换为指定格式的字典列表？

需要将稀疏DataFrame转换为字典列表，要求以列名为键，列值为对应值（部分列值为字符串形式的元组），用于后续键值对比操作。但现有代码触发ValueError，具体如下：

mydict={'code0': {0: 'nan', 1: " '40'", 2: " '98'", 3: " '98'", 4: " '52'", 5: " '52'", 6: " '52'", 7: " '52'", 8: " '40'", 9: " '58'"}, 'code1': {0: "  ('VA','HC','NIH','SAP','AUS','HOL','ATT','COL','UCL')", 1: 'nan', 2: " ('ATT','NC')", 3: " ('ATT','VA','NC')", 4: " 'NC'", 5: " 'NC'", 6: " 'NC'", 7: " 'NC'", 8: " 'VA'", 9: " 'CE'"}, 'code2': {0: 'nan', 1: 'nan', 2: " ('103','104','105','106','31')", 3: " ('104','105','106','31')", 4: " '109'", 5: " '109'", 6: " '109'", 7: " '109'", 8: " '11'", 9: " ('109')"}, 'code3': {0: 'nan', 1: " '518'", 2: " '810'", 3: 'nan', 4: " ('610','620','682','642','621','611')", 5: " ('396','340','394','393','240')", 6: " ('612','790','110')", 7: " ('730','320','350','379','812','374')", 8: " ('113','174','131','115')", 9: " ('423','114')"}, 'code4': {0: 'nan', 1: 'nan', 2: " 'computer science'", 3: "  'computer science'", 4: " 'biology'", 5: " 'biology'", 6: "biology'", 7: "biology'", 8: 'nan', 9: 'nan'}, 'code5': {0: 'nan', 1: 'nan', 2: 'nan', 3: 'nan', 4: 'nan', 5: " ('12','18')", 6: " ('12','16','18','19')", 7: " ('12','18','19')", 8: " ('11','19','31')", 9: " '31'"}, 'code6': {0: 'nan', 1: " '594'", 2: 'nan', 3: 'nan', 4: " ('712','479','297','639','452','172')", 5: 'nan', 6: " ('285','295','236','239','269','284','237')", 7: 'nan', 8: " ('164','157','388','158')", 9: " ('372','238')"}, 'rules_desc': {0: 'rules1', 1: 'rules2', 2: 'rules2', 3: 'rules2', 4: 'rules2', 5: 'rules2', 6: 'rules2', 7: 'rules2', 8: 'rules2', 9: 'rules2'}}

mydf=pd.DataFrame.from_dict(mydict)
cols = mydf.columns.values
res=[",".join("{}:{}".format(*t) for t in zip(cols, row)) for _, row in mydf[cols].iterrows()]
res=[list(s.split('""')) for s in res]

intconv = lambda x: (x[0], int(x[1]))
for s in res:
    b = dict([i.split(':') for i in s])
    final = dict((k, int(v)) for k, v in b.items())
    print(final)

报错信息：

--------------------------------------------------------------------------- ValueError Traceback (most recent call
last) Input In [60], in <cell line: 9>()
8 intconv = lambda x: (x[0], int(x[1]))
9 for s in res:
---> 10 b = dict([i.split(':') for i in s])
11 final = dict((k, int(v)) for k, v in b.items())
12 print(final)

ValueError: dictionary update sequence element #0 has length 9; 2 is required

期望输出格式：

output_dict=[
    {'code0':'nan', 'code1': ('VA','HC','NIH','SAP','AUS','HOL','ATT','COL','UCL'), 'code2':'nan', 'code3':'nan', 'code4':'nan', 'code5':'nan', 'code6':'nan','rules_desc': 'rules1'},
    {'code0': 40, 'code1':'nan','code2':'nan','code3':518, 'code4':'nan', 'code5':'nan','code6':594, 'rules_desc': 'rules2'},
    ...
]

原代码的错误在于通过字符串拼接再拆分的方式处理数据，逻辑混乱导致拆分后的元素不符合字典构造要求；同时没有处理字符串形式的元组、单引号包裹的数值/字符串，以及nan的转换逻辑。

正确实现思路是直接遍历DataFrame的每一行，对每个单元格的内容进行解析：

1. 使用ast.literal_eval解析字符串形式的元组或单值，自动处理引号和括号；
2. 对解析后的值进行类型转换，比如将数字字符串转为整数；
3. 保留nan的字符串形式（或根据需求转为None）；
4. 兼容格式错误的情况（比如code4中的biology'缺少左引号）。

具体代码：

import pandas as pd
import ast

# 原始数据
mydict={'code0': {0: 'nan', 1: " '40'", 2: " '98'", 3: " '98'", 4: " '52'", 5: " '52'", 6: " '52'", 7: " '52'", 8: " '40'", 9: " '58'"}, 'code1': {0: "  ('VA','HC','NIH','SAP','AUS','HOL','ATT','COL','UCL')", 1: 'nan', 2: " ('ATT','NC')", 3: " ('ATT','VA','NC')", 4: " 'NC'", 5: " 'NC'", 6: " 'NC'", 7: " 'NC'", 8: " 'VA'", 9: " 'CE'"}, 'code2': {0: 'nan', 1: 'nan', 2: " ('103','104','105','106','31')", 3: " ('104','105','106','31')", 4: " '109'", 5: " '109'", 6: " '109'", 7: " '109'", 8: " '11'", 9: " ('109')"}, 'code3': {0: 'nan', 1: " '518'", 2: " '810'", 3: 'nan', 4: " ('610','620','682','642','621','611')", 5: " ('396','340','394','393','240')", 6: " ('612','790','110')", 7: " ('730','320','350','379','812','374')", 8: " ('113','174','131','115')", 9: " ('423','114')"}, 'code4': {0: 'nan', 1: 'nan', 2: " 'computer science'", 3: "  'computer science'", 4: " 'biology'", 5: " 'biology'", 6: "biology'", 7: "biology'", 8: 'nan', 9: 'nan'}, 'code5': {0: 'nan', 1: 'nan', 2: 'nan', 3: 'nan', 4: 'nan', 5: " ('12','18')", 6: " ('12','16','18','19')", 7: " ('12','18','19')", 8: " ('11','19','31')", 9: " '31'"}, 'code6': {0: 'nan', 1: " '594'", 2: 'nan', 3: 'nan', 4: " ('712','479','297','639','452','172')", 5: 'nan', 6: " ('285','295','236','239','269','284','237')", 7: 'nan', 8: " ('164','157','388','158')", 9: " ('372','238')"}, 'rules_desc': {0: 'rules1', 1: 'rules2', 2: 'rules2', 3: 'rules2', 4: 'rules2', 5: 'rules2', 6: 'rules2', 7: 'rules2', 8: 'rules2', 9: 'rules2'}}

mydf = pd.DataFrame.from_dict(mydict)

def parse_value(val):
    # 处理nan
    if val.strip() == 'nan':
        return 'nan'
    # 补全缺失的左引号
    val = val.strip()
    if val.startswith("'") == False and val.endswith("'"):
        val = f"'{val}"
    # 处理('xxx')这种单元素"元组"，转为单值
    elif val.startswith("('") and val.endswith("')") and len(val) == 5:
        return val[2:-2]
    try:
        # 解析元组或单值
        parsed = ast.literal_eval(val)
        # 字符串尝试转整数，失败则保留原字符串
        if isinstance(parsed, str):
            try:
                return int(parsed)
            except ValueError:
                return parsed
        # 元组元素尝试转整数
        elif isinstance(parsed, tuple):
            new_tuple = []
            for item in parsed:
                try:
                    new_tuple.append(int(item))
                except ValueError:
                    new_tuple.append(item)
            return tuple(new_tuple)
        return parsed
    except:
        # 解析失败返回处理后的原字符串
        return val.strip(" '")

# 生成字典列表
output_dict = []
for _, row in mydf.iterrows():
    row_dict = {}
    for col in mydf.columns:
        row_dict[col] = parse_value(row[col])
    output_dict.append(row_dict)

# 打印结果
for d in output_dict:
    print(d)

代码关键点说明 #

• parse_value函数统一处理所有单元格值，覆盖nan、格式错误、元组解析、数字转换等场景；
• 直接遍历DataFrame行列构造字典，逻辑清晰，避免字符串拼接拆分的混乱；
• 对解析失败的情况做兼容，确保程序不会因格式错误崩溃。

内容的提问来源于stack exchange，提问作者Hamilton