Python：如何处理DataFrame中以字符串存储的嵌套字典？

Let's break this down into two straightforward steps to turn your sensor data into a usable, structured DataFrame:

Step 1: Convert String Values to Actual Python Dictionaries #

Your streams column stores nested dictionaries as raw strings—so first we need to parse them into real Python dictionary objects. The safest way to do this is using ast.literal_eval() (it avoids the security risks of using eval() directly):

import ast
import pandas as pd

# Parse each string in the streams column into a proper dictionary
df['streams_parsed'] = df['streams'].apply(ast.literal_eval)

Step 2: Expand the Nested Dictionaries into a Structured DataFrame #

Now that we have actual dictionaries, we can use Pandas' json_normalize() function to flatten the nested structure. You have two common options depending on your desired output:

Option 1: Each Sensor as a Separate Row #

If you want every sensor's data (AirBeam-PM, AirBeam-RH, etc.) to be its own row in the final DataFrame:

# Extract the sensor data dictionaries from each parsed entry, then explode into individual rows
sensor_rows = df['streams_parsed'].apply(lambda x: list(x.values())).explode()

# Flatten each sensor's dictionary into columns
final_df = pd.json_normalize(sensor_rows)

Option 2: Flatten Sensors into Prefixed Columns #

If you prefer to keep each original row and have sensor-specific columns (e.g., AirBeam-PM.average_value, AirBeam-RH.unit_symbol):

final_df = pd.json_normalize(df['streams_parsed'])

Quick Output Preview #

For Option 2, your final DataFrame will include columns like:

• AirBeam-PM.session_id
• AirBeam-PM.average_value
• AirBeam-RH.unit_symbol
• ...and all other fields from each sensor's dictionary, prefixed with the sensor name for clarity.

This setup gives you a fully analyzable DataFrame ready for further sensor data processing!

内容的提问来源于stack exchange，提问作者Nathan Furnal