如何将Pandas DataFrame中JSON字符串列拆分为多列多行?
问题描述
现有一个Pandas DataFrame,其中json列存储的是JSON格式的字符串(数组形式),需要将该列拆分为对应的多列和多行,解决原JSON字符串过长存入SQL时被截断的问题。当前DataFrame与目标DataFrame示例如下:
当前DataFrame
| Department | Value | json |
|---|---|---|
| 1 | A | [{"employeeID":123,"name":"Jenny"}, {"employeeID":456,"name":"Mike"}, {"employeeID":789,"name":"Ricky"}] |
| 2 | B | [{"employeeID":735,"name":"Todd", "badgeNo":84639}, {"employeeID":223,"name":"Greg", "badgeNo":93649}] |
| 3 | C | [] |
| 4 | D | [{"employeeID":947,"name":"Cardi"}, {"employeeID":284,"name":"Tom"}] |
目标DataFrame
| Department | Value | employeeID | name | badgeNo |
|---|---|---|---|---|
| 1 | A | 123 | Jenny | |
| 1 | A | 456 | Mike | |
| 1 | A | 789 | Ricky | |
| 2 | B | 735 | Todd | 84639 |
| 2 | B | 223 | Greg | 93649 |
| 3 | C | |||
| 4 | D | 947 | Cardi | |
| 4 | D | 284 | Tom |
Python实现方案
通过以下步骤完成转换:
- 导入
pandas和json库 - 将
json列的字符串解析为Python对象(列表/字典) - 把列表类型的
json列拆分为多行(空列表保留原行) - 将字典类型的字段展开为多列
- 合并原DataFrame的非
json列与展开后的列
完整代码示例:
import pandas as pd import json # 构造示例DataFrame df = pd.DataFrame({ 'Department': [1, 2, 3, 4], 'Value': ['A', 'B', 'C', 'D'], 'json': [ '[{"employeeID":123,"name":"Jenny"}, {"employeeID":456,"name":"Mike"}, {"employeeID":789,"name":"Ricky"}]', '[{"employeeID":735,"name":"Todd", "badgeNo":84639}, {"employeeID":223,"name":"Greg", "badgeNo":93649}]', '[]', '[{"employeeID":947,"name":"Cardi"}, {"employeeID":284,"name":"Tom"}]' ] }) # 解析JSON字符串为Python对象 df['json'] = df['json'].apply(json.loads) # 拆分JSON列表为多行,空列表保留原行 df_exploded = df.explode('json', ignore_index=True) # 将字典展开为多列,缺失字段用NaN填充 df_normalized = pd.json_normalize(df_exploded['json']) # 合并原列与展开后的列 result = pd.concat([df_exploded.drop('json', axis=1), df_normalized], axis=1) # 把NaN替换为空字符串,匹配目标格式(可选) result = result.fillna('') print(result)
代码说明
json.loads:将JSON字符串转换为Python可处理的列表/字典对象explode('json', ignore_index=True):把每个列表元素拆成单独行,重置索引避免重复pd.json_normalize():自动将字典的键转为列,缺失字段用NaN填充fillna(''):将NaN替换为空字符串,和目标示例格式一致,不需要可省略
内容的提问来源于stack exchange,提问作者Jwest
相关产品推荐
相关产品推荐

