RNN训练数据转换构造后无法获取数据集最后一行是什么原因

问题复现 #

执行如下代码构造RNN训练样本时，X_train[-1][-1]返回的是训练集倒数第二行，而非预期的最后一行，将X_train展平后也找不到最后一行数据：

import numpy as np
train = np.array([[-0.35056551,-0.0486274,0.13748505,-0.39753398,-1.96119178,-2.06431894,-1.87555811,1.12317278,2.05828838,1.42899677],[-0.38915169,-0.34838332,-0.39534301,-0.35003678,-1.87457696,-1.89019837,-1.93096233,1.20455178,4.05423391,1.39762687],[-0.25642087,-0.38685423,-0.34794132,-0.39931924,-1.93378219,-1.9458735,-1.94236835,1.18646752,0.39861687,1.25960107],[0.0835339,-0.2545199,-0.39712468,-0.26289869,-1.88608916,-1.9573353,-1.83427558,1.02190142,-0.38842136,1.28155967],[0.66626562,0.0844194,-0.26097848,0.28261131,-1.69805802,-1.84871399,-1.60016699,0.85552685,1.99962579,1.28155967],[0.60999256,0.66541044,0.28343446,0.68276927,-1.43985782,-1.61346066,-1.50402534,0.85010147,1.73335433,1.46820851],[0.49369643,0.60930547,0.68278767,0.73776688,-1.46068961,-1.5168489,-1.44155941,0.59330596,0.93824727,1.48546164],[0.549433,0.49335673,0.73767468,0.56223987,-1.62459964,-1.45407753,-1.59527864,0.53543661,-0.33357511,1.5482007,],[0.30969586,0.54892681,0.56250066,0.40367636,-1.48535825,-1.60854841,-1.51434419,0.38714631,0.88143833,1.65799445],[0.27235858,0.30990578,0.40425603,0.45099527,-1.52592343,-1.52721821,-1.45296641,0.47033376,-0.09347284,1.65172062],[0.48244204,0.27268004,0.45147978,0.25493303,-1.52373167,-1.46554031,-1.58169903,0.5010769,-0.74661207,1.70348038],[0.34613823,0.48213596,0.25581184,0.45635233,-1.48809921,-1.5949024,-1.50945653,0.28949186,-0.60071887,1.74426048],[0.18285884,0.3462393,0.45682608,0.26600367,-1.53469551,-1.52230665,-1.5284821,0.27502452,0.28347246,1.78190422],[-0.08421009,0.18344764,0.26686022,0.06833588,-1.56551451,-1.54142526,-1.63372597,0.63128294,-0.28538019,1.76465109],[-0.50669891,-0.08282352,0.06958995,-0.34789348,-1.70640182,-1.64718374,-1.8818389,1.55538516,0.284999,2.1144218,],[-0.46114594,-0.50405033,-0.34580233,-0.38021489,-1.88966568,-1.89650987,-1.86057212,1.85196542,1.11379888,2.19598199],[-0.34431332,-0.45863343,-0.37805875,-0.30557473,-1.9441498,-1.87513911,-1.90528652,1.73984364,0.32447085,2.28224877],[-0.87184332,-0.3421498,-0.30356869,-0.61270217,-1.72126128,-1.92007215,-1.81803857,1.37092634,-0.24678065,2.34028237],[-0.84558289,-0.86810402,-0.61007847,-0.91536598,-1.79665781,-1.83239758,-1.79131844,1.82122263,1.72713915,2.02344993],[-0.67950941,-0.84192203,-0.9121336,-0.67860981,-1.78014739,-1.80554679,-1.7051605,1.42156185,2.40753796,2.158339,],[-0.81766348,-0.67634463,-0.67585357,-0.818503,-1.57211834,-1.71896756,-1.62881845,1.5698525,0.05416498,1.91208782],[-0.80901793,-0.81408602,-0.81546542,-0.72055958,-1.5181851,-1.64225222,-1.44723252,1.17923402,0.3616529,1.84778039],[-0.58368381,-0.80546629,-0.71771898,-0.66204562,-1.26227659,-1.45977839,-1.20784464,1.13944858,-0.51217979,1.84150618],[-0.64456549,-0.58080527,-0.65932269,-0.50360803,-1.20614151,-1.21921996,-1.17021862,1.00020024,-0.02368835,1.98266946],[-0.77533446,-0.64150509,-0.50120374,-0.52503319,-1.28043796,-1.18140995,-1.2727358,0.74702165,-0.89381374,2.07834643],[-0.92015355,-0.77188344,-0.5225858,-0.83344632,-1.46039906,-1.28442842,-1.45704757,1.09062119,-0.6255796,1.94502572]])
X_train = []
Y_train = []
timesteps = 20
for i in range(timesteps,train.shape[0]):
    X_train.append(train[i-timesteps:i])
    Y_train.append(train[i][0])
X_train,Y_train = np.array(X_train),np.array(Y_train)
print(X_train[-1][-1])

实际返回结果：

[-0.77533446,-0.64150509,-0.50120374,-0.52503319,-1.28043796,-1.18140995,-1.2727358,0.74702165,-0.89381374,2.07834643]

根本原因 #

问题出在Python切片的左闭右开规则，和时序样本构造的逻辑匹配问题：

1. 你的训练集train共26行，索引范围为0~25。
2. Python切片a[start:end]仅包含start到end-1位置的元素，不包含end位置的元素。
3. 循环for i in range(timesteps,train.shape[0])中，i的取值为20、21、22、23、24、25，i的最大值为25。
4. 当i取最大值25时，输入特征切片为train[25-20:25] = train[5:25]，仅包含索引5到24的行（也就是倒数第二行及之前的内容），索引25的最后一行不会被放进X_train，仅它的第0列值被取为最后一个样本的标签。

结论 #

数据集的最后一行没有传入RNN的输入特征集X_train，仅最后一行的第0列被作为标签使用。

修复方案 #

根据你的任务目标调整逻辑即可：

• 如果是最常见的单步预测任务（用前20步数据预测第21步的目标值）：现有X的构造逻辑是正确的。单步预测的输入特征本来就是标签行之前的timesteps行数据，数据集最后一行没有对应的下一行标签，本身就不应该出现在训练输入中，你之前的预期是认知偏差。
• 如果是序列到序列、序列标注类任务，需要输入窗口覆盖最后一行数据，需要调整循环范围和标签取值逻辑，注意避免索引越界，示例如下：

timesteps = 20
X_train = []
Y_train = []
# 调整循环上界，让切片结束索引可以取到train.shape[0]，覆盖最后一行
for i in range(timesteps, train.shape[0] + 1):
    feature_win = train[i-timesteps:i]
    X_train.append(feature_win)
    # 标签根据任务需求从窗口内取，禁止再取train[i]避免越界
    Y_train.append(feature_win[-1][1:]) # 示例：取窗口最后一行除第0列外的特征作为标签

你提到的生成器构造数据集的方案完全可行，只要注意切片的左闭右开规则，就能避免数据遗漏或错位问题。

内容的提问来源于stack exchange，提问作者agamveer.franz