如何将嵌套字典列表转换为Polars DataFrame并补全缺失列？

一、修改原有循环代码补充缺失列 #

你的原有循环只提取了date、credit_score、spend三个字段，没有将name、fame、average_spend对应到每一行。只需在循环中把这些标量值重复对应到每个日期行即可：

import polars as pl

list_sample = [
    {
        'name': 'A',
        'fame': 0,
        'data': {
            'date': ['2021-01-01', '2021-02-01', '2021-03-01'],
            'credit_score': [800, 890, 895],
            'spend': [1500, 25000, 2400],
            'average_spend': 5000
        }
    },
    {
        'name': 'B',
        'fame': 1,
        'data': {
            'date': ['2022-01-01', '2022-02-01', '2022-03-01'],
            'credit_score': [2800, 390, 8900],
            'spend': [15000, 5000, 400],
            'average_spend': 3000
        }
    }
]

rows = []

for item in list_sample:
    name = item['name']
    fame = item['fame']
    avg_spend = item['data']['average_spend']
    date_list = item['data']['date']
    credit_scores = item['data']['credit_score']
    spends = item['data']['spend']
    # 将标量值重复对应到每个日期行
    rows.extend(zip(
        [name]*len(date_list),
        [fame]*len(date_list),
        date_list,
        credit_scores,
        spends,
        [avg_spend]*len(date_list)
    ))

df = pl.DataFrame(rows, schema=["name", "fame", "date", "credit_score", "spend", "average_spend"])

二、更简便的Polars内置方法实现 #

Polars提供了unnest和explode方法，可以完全替代手动循环，代码更简洁且效率更高：

import polars as pl

list_sample = [
    {
        'name': 'A',
        'fame': 0,
        'data': {
            'date': ['2021-01-01', '2021-02-01', '2021-03-01'],
            'credit_score': [800, 890, 895],
            'spend': [1500, 25000, 2400],
            'average_spend': 5000
        }
    },
    {
        'name': 'B',
        'fame': 1,
        'data': {
            'date': ['2022-01-01', '2022-02-01', '2022-03-01'],
            'credit_score': [2800, 390, 8900],
            'spend': [15000, 5000, 400],
            'average_spend': 3000
        }
    }
]

# 1. 从字典列表创建初始DataFrame
# 2. 用unnest展开`data`结构体的所有字段为单独列
# 3. 用explode将数组类型的列展开，标量列自动广播到每一行
df = pl.DataFrame(list_sample).unnest("data").explode(["date", "credit_score", "spend"])

最终生成的DataFrame会包含所有需要的列：

shape: (6, 6)
┌──────┬──────┬────────────┬──────────────┬───────┬───────────────┐
│ name ┆ fame ┆ date       ┆ credit_score ┆ spend ┆ average_spend │
│ ---  ┆ ---  ┆ ---        ┆ ---          ┆ ---   ┆ ---           │
│ str  ┆ i64  ┆ str        ┆ i64          ┆ i64   ┆ i64           │
╞══════╪══════╪════════════╪══════════════╪═══════╪═══════════════╡
│ A    ┆ 0    ┆ 2021-01-01 ┆ 800          ┆ 1500  ┆ 5000          │
│ A    ┆ 0    ┆ 2021-02-01 ┆ 890          ┆ 25000 ┆ 5000          │
│ A    ┆ 0    ┆ 2021-03-01 ┆ 895          ┆ 2400  ┆ 5000          │
│ B    ┆ 1    ┆ 2022-01-01 ┆ 2800         ┆ 15000 ┆ 3000          │
│ B    ┆ 1    ┆ 2022-02-01 ┆ 390          ┆ 5000  ┆ 3000          │
│ B    ┆ 1    ┆ 2022-03-01 ┆ 8900         ┆ 400   ┆ 3000          │
└──────┴──────┴────────────┴──────────────┴───────┴───────────────┘

内容的提问来源于stack exchange，提问作者r ram