如何从4D xarray数据集生成多级索引矩阵?
问题描述
现有一个4D xarray数据集ds,结构如下:
<xarray.Dataset> Dimensions: (lat: 60, lon: 78, time: 216, pres: 395) Coordinates: * lat (lat) float32 0.5 1.5 2.5 3.5 4.5 5.5 ... 55.5 56.5 57.5 58.5 59.5 * lon (lon) float32 -45.5 -44.5 -43.5 -42.5 ... -69.5 -75.5 -74.5 -76.5 * time (time) float32 7.32e+05 7.32e+05 7.32e+05 ... 7.385e+05 7.385e+05 * pres (pres) float64 2.5 7.5 12.5 17.5 ... 1.962e+03 1.968e+03 1.972e+03 Data variables: var (pres, lat, lon, time) float64 2.03e+03 2.03e+03 ... nan nan
需要将其转换为特定结构的pandas DataFrame:
id time pres param 20.5-70.5 20.5-71.5 20.5-72.5 0 0 0 var 2085 2073 2057 1 0 1 var 2114 2156 2054 2 0 2 var 2039 2006 2179 3 1 0 var 2199 2144 2033 4 1 1 var 2056 2102 2191 5 1 2 var 2062 2033 2052 6 2 0 var 2001 2153 2170 7 2 1 var 2187 2120 2100 8 2 2 var 2138 2076 2002
要求:
time与pres作为行标识(可转为列)- 新增
param列(后续支持多变量扩展) - 每个经纬度对格式化为
lat-lon字符串作为列名,对应time+pres组合下的var值
之前尝试stacked = ds.stack(coordinates=["lat", "lon"])后调用stacked.to_dataframe(),但经纬度对被拆分为多级索引,不符合需求。
解决方案
单变量转换步骤
import pandas as pd import xarray as xr # 1. 堆叠lat和lon为单个pixel维度,生成MultiIndex stacked = ds.stack(pixel=("lat", "lon")) # 2. 将pixel的MultiIndex转换为"lat-lon"格式的字符串标签 stacked = stacked.assign_coords( pixel=[f"{lat:.1f}-{lon:.1f}" for lat, lon in stacked.pixel.values] ) # 3. 将pixel维度转换为列(unstack操作),此时行索引为pres和time unstacked = stacked.unstack("pixel") # 4. 转换为DataFrame并扁平化列名(原列名是(var, pixel),提取pixel部分) df = unstacked.to_dataframe() df.columns = [col[1] for col in df.columns] # 5. 添加param列,当前变量为var,所以全填充"var" df["param"] = "var" # 6. 重置索引,将pres和time从索引转为列,同时添加id行号列 df = df.reset_index() df["id"] = df.index # 7. 调整列顺序,匹配目标结构 df = df[["id", "time", "pres", "param"] + [col for col in df.columns if col not in ["id", "time", "pres", "param"]]]
多变量扩展方案
如果后续需要添加多个变量(如var1、var2),可以循环处理每个变量后合并:
df_list = [] for var_name in ds.data_vars: # 对单个变量重复核心转换步骤 var_stacked = ds[var_name].stack(pixel=("lat", "lon")) var_stacked = var_stacked.assign_coords( pixel=[f"{lat:.1f}-{lon:.1f}" for lat, lon in var_stacked.pixel.values] ) var_df = var_stacked.unstack("pixel").to_dataframe(name=var_name) var_df.columns = [col for col in var_df.columns] var_df["param"] = var_name var_df = var_df.reset_index() df_list.append(var_df) # 合并所有变量的DataFrame final_df = pd.concat(df_list, ignore_index=True) final_df["id"] = final_df.index # 调整列顺序 final_df = final_df[["id", "time", "pres", "param"] + [col for col in final_df.columns if col not in ["id", "time", "pres", "param"]]]
关键说明
stack(pixel=("lat", "lon"))将二维经纬度合并为单个维度,避免后续拆分- 手动格式化
pixel坐标为字符串,确保unstack后列名是lat-lon的格式 unstack("pixel")将经纬度维度转为列,正好匹配目标结构的列需求
内容的提问来源于stack exchange,提问作者ldlg
相关产品推荐
相关产品推荐

