如何用Pandas实现三维数组的宽表转长表(类似stack)
二维数组转长表示例
首先创建测试数据集:
import numpy as np from scipy.stats import norm import pandas as pd np.random.seed(10) n=3 space= norm(20, 5).rvs(n) time= norm(10,2).rvs(n) values = np.kron(space, time).reshape(n,n) + norm(1,1).rvs([n,n])
输出:
array([[267.39784458, 300.81493866, 229.19163206], [236.1940266 , 266.49469945, 204.01294305], [122.55912977, 140.00957047, 106.28339745]])
将其转换为Pandas宽表:
space_names = ['A','B','C'] time_names = [2000,2001,2002] df = pd.DataFrame(values, index=space_names, columns=time_names) df
输出:
2000 2001 2002 A 267.397845 300.814939 229.191632 B 236.194027 266.494699 204.012943 C 122.559130 140.009570 106.283397
这是典型的宽表结构,每个观测值由两个变量作为坐标标识。我们可通过Pandas的.stack()方法将其转换为整洁的长表:
df.columns.name = 'time' df.index.name = 'space' df.stack().rename('value').reset_index()
输出:
space time value 0 A 2000 267.397845 1 A 2001 300.814939 2 A 2002 229.191632 3 B 2000 236.194027 4 B 2001 266.494699 5 B 2002 204.012943 6 C 2000 122.559130 7 C 2001 140.009570 8 C 2002 106.283397
三维数组转长表需求
现在需要对三维数组执行类似操作:每个时空组合对应2个观测值,测试数据如下:
s = 3 t = 4 r = 2 space_mus = norm(20, 5).rvs(s) time_mus = norm(10,2).rvs(t) values = np.kron(space_mus, time_mus) values = values.repeat(r).reshape(s,t,r) + norm(0,1).rvs([s,t,r]) values
输出:
array([[[286.50322099, 288.51266345], [176.64303485, 175.38175877], [136.01675917, 134.44328617]], [[187.07608546, 185.4068411 ], [112.86398438, 111.983463 ], [ 85.99035255, 86.67236986]], [[267.66833894, 269.45295404], [162.30044715, 162.50564386], [124.6374401 , 126.2315447 ]]])
如何将该三维数组转换为类似上述的整洁长表结构?
现有不够优雅的实现
本人实现了一种可用但不够Pythonic的方法,现寻求更简洁优雅的解决方案:
labels = ['{}{}{}'.format(i,j,k) for i in range(s) for j in range(t) for k in range(r)] #space, time, repetition def flatten3d(k): return [i for l in k for s in l for i in s] value_series = pd.Series(flatten3d(values)).rename('y') split_labels= [[i for i in l] for l in labels] df = pd.DataFrame(split_labels, columns=['s','t','r']) pd.concat([df, value_series], axis=1)
输出:
s t r y 0 0 0 0 266.2408815208753 1 0 0 1 266.13662442609433 2 0 1 0 299.53178992512954 3 0 1 1 300.13941632567605 4 0 2 0 229.39037800681405 5 0 2 1 227.22227496248507 6 0 3 0 281.76357915411995 7 0 3 1 280.9639352062619 8 1 0 0 235.8137644198259 9 1 0 1 234.23202459516452 10 1 1 0 265.19681013560034 11 1 1 1 266.5462102589883 12 1 2 0 200.730100791878 13 1 2 1 199.83217739700535 14 1 3 0 246.54018839875374 15 1 3 1 248.5496308586532 16 2 0 0 124.90916276929234 17 2 0 1 123.64788669199066 18 2 1 0 139.65391860786775 19 2 1 1 138.08044561039517 20 2 2 0 106.45276370157518 21 2 2 1 104.78351933651582 22 2 3 0 129.86043618610572 23 2 3 1 128.97991481257253
内容的提问来源于stack exchange,提问作者StackyGuy
相关产品推荐
相关产品推荐

