如何在Pandas中对字典列表正确使用pd.json_normalize?
问题分析与解决
你的错误并非因为pd.json_normalize()不支持字典列表——实际上它本身就支持传入字典列表作为输入。报错的根源是横向拼接的两个DataFrame行数不匹配:
pd.DataFrame(data)会把整个原始数据转成只有1行的DataFrame(仅包含examples一个列,值为整个列表)pd.json_normalize(data['examples'])会生成2行的DataFrame(对应两个website条目)
两者行数不一致,执行pd.concat(..., axis=1)时自然会报错。
下面根据不同需求给出正确的处理方式:
需求1:保留每个website对应的完整df列表
如果只是想把examples里的每个条目规整为DataFrame的一行,直接用pd.json_normalize()处理data['examples']即可,无需多余的concat操作:
import pandas as pd def filter_data(data, name): # 直接处理字典列表,生成包含website和df列的DataFrame normalized_df = pd.json_normalize(data['examples']) return normalized_df # 测试调用 data = { "examples": [ { "website": "info", "df": [ {"Question": "What?", "Answers": []}, {"Question": "how?", "Answers": []}, {"Question": "Why?", "Answers": []} ] }, { "website": "info2", "df": [ {"Question": "What?", "Answers": ["example answer1"]}, {"Question": "how?", "Answers": ["example answer1"]}, {"Question": "Why?", "Answers": ["example answer1"]} ] } ] } result = filter_data(data, "test") print(result)
输出结果:
website df 0 info [{'Question': 'What?', 'Answers': []}, {'Quest... 1 info2 [{'Question': 'What?', 'Answers': ['example an...
需求2:展开嵌套的Question-Answers并关联website
如果需要把每个df里的Question-Answers对单独成行,同时关联对应的website,可以利用pd.json_normalize()的record_path和meta参数,一次性展开嵌套结构:
import pandas as pd def filter_data(data, name): # 展开嵌套的df列表,同时保留上层的website字段 normalized_df = pd.json_normalize( data['examples'], record_path='df', # 指定要展开的嵌套列表字段 meta='website' # 指定要保留的上层关联字段 ) return normalized_df # 测试调用 result = filter_data(data, "test") print(result)
输出结果:
Question Answers website 0 What? [] info 1 how? [] info 2 Why? [] info 3 What? [example answer1] info2 4 how? [example answer1] info2 5 Why? [example answer1] info2
内容的提问来源于stack exchange,提问作者Serkan Gün
相关产品推荐
相关产品推荐

