如何提取含DynamoDB字典结构的DataFrame中的值并解决操作报错

方案1：基于现有client代码修改，无需切换接口 #

你之前的报错是因为pd.json_normalize返回DataFrame结构，无法直接赋值给新DataFrame的单列。可以直接在转DataFrame前对原始返回数据做预处理，提取每个字典的实际值：

import boto3
import pandas as pd

client = boto3.client('dynamodb')

def dump_table(table_name):
    results = []
    last_evaluated_key = None
    while True:
        if last_evaluated_key:
            response = client.scan(
                TableName=table_name,
                ExclusiveStartKey=last_evaluated_key
            )
        else: 
            response = client.scan(TableName=table_name)
        last_evaluated_key = response.get('LastEvaluatedKey')
        results.extend(response['Items'])
        if not last_evaluated_key:
            break
    return results

# 提取每个字段的实际值
data = dump_table('my_table_name')
processed_data = []
for item in data:
    # DynamoDB每个字段的字典仅包含1个类型key，直接取第一个value即可
    processed_item = {col: next(iter(val.values())) for col, val in item.items()}
    processed_data.append(processed_item)

df = pd.DataFrame(processed_data)

方案2：切换为boto3 resource实现，无需手动处理类型 #

boto3的resource接口会自动解析DynamoDB的类型标识，直接返回字段实际值，实现代码如下：

import boto3
import pandas as pd

# 初始化resource接口
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('my_table_name')

def dump_table(table):
    results = []
    last_evaluated_key = None
    while True:
        if last_evaluated_key:
            response = table.scan(ExclusiveStartKey=last_evaluated_key)
        else:
            response = table.scan()
        last_evaluated_key = response.get('LastEvaluatedKey')
        results.extend(response['Items'])
        if not last_evaluated_key:
            break
    return results

data = dump_table(table)
df = pd.DataFrame(data)

内容的提问来源于stack exchange，提问作者Bushmaster