TensorFlow时间序列预测输出值范围异常问题求助

数据集与参数 #

我有一个包含36619个样本、6个特征的价格数据集，各特征分布差异显著。使用TensorFlow 2.8.2，定义核心参数如下：

WINDOW_JUMP = 24
WINDOW_SIZE = 24*7
SHIFT = 1
TARGET = 1
BATCH_SIZE = 128
SHUFFLE_BUFFER = 10000
SERIES_SHAPE = [WINDOW_SIZE,6]

其中WINDOW_SIZE使单样本形状为168×6，目标是预测提前24个时间步（即WINDOW_JUMP）的数值。

窗口数据集生成函数 #

编写了以下函数，用于适配Normalization层并生成TensorFlow数据集：

def windowed_dataset(series,
    window_size, 
    window_jump, 
    target, 
    batch_size,
    shuffle_buffer,
    dim_target = 0,
    shift=1,
    processing = None):
    """Generates dataset windows

    Args:
      series (array of float) - contains the values of the time series
      window_size (int) - the number of time steps to include in the feature
      window_jump (int) - number of time steps ahead to predict
      target (int) - number of targets to predict from window_size to window_jump
      batch_size (int)
      shuffle_buffer(int) - buffer size to use for the shuffle method
      dim_target (int) - Case of a multivariate dataset, the number of the column from which to extract the targets
      shift (int) - jump between windows
      Processing (Keras.layers.preprocessing Object) - If passed, Preprocessing layer to adapt

    Returns:
      dataset (TF Dataset) - TF Dataset containing time windows and targets
    """
    print('--> Generate a TF Dataset from the series values')
    dataset = tf.data.Dataset.from_tensor_slices(series)

    if processing != None:
      print('\t --> Adapting the preprocessed layer to the data')
      processed = dataset.window(window_size, shift=shift, drop_remainder=True)
      processed = processed.flat_map(lambda window: window.batch(window_size))
      processing.adapt(processed)

    print('--> Window the data but only take those with the specified size')
    dataset = dataset.window(window_size + window_jump, shift=shift, drop_remainder=True)
    
    print('--> Flatten the windows by putting its elements in a single batch')
    dataset = dataset.flat_map(lambda window: window.batch(window_size + window_jump))
    
    print('--> Create tuples with features and labels')
    dataset = dataset.map(lambda window: (window[:-window_jump], window[-target:][:,dim_target]))
    
    print('--> Shuffle the windows')
    dataset = dataset.shuffle(shuffle_buffer)
    
    print('--> Create batches of windows')
    dataset = dataset.batch(batch_size).prefetch(1)
    
    if processing != None:
      print('Returning dataset and adapted layer')
      return dataset,processing
    else:
      print('Returning dataset')
      return dataset

数据划分与处理 #

将数据集按90%/5%/5%划分为训练集、验证集、测试集，已确认各集目标值均在合理范围内。处理训练集的代码如下：

# Train set
train_series, normalizer_layer = windowed_dataset(train_set,
                                WINDOW_SIZE,WINDOW_JUMP,TARGET,BATCH_SIZE,SHUFFLE_BUFFER,
                                processing= tf.keras.layers.Normalization(input_shape=SERIES_SHAPE))

模型构建与训练 #

构建并训练模型的代码如下：

# Train the model
def compile_fit_model(model,epochs,train_series,validation_series,lr_schedule):
  # Initialize the optimizer
  print('\nCreating optimizer with scheduler')
  optimizer = tf.keras.optimizers.Adam(lr_schedule)

  # Set the training parameters
  model.compile(loss='mse', optimizer=optimizer, metrics=['mae'])

  print('\nFitting the model')
  history = model.fit(train_series, epochs=epochs)#,validation_data=validation_series)

  return history

def get_uncompile_model(input_shape,norm=None):
    '''Model with the changes proposed by Pranav Raikote'''
    model = tf.keras.models.Sequential()
    if norm != None:
        model.add(norm)
    model.add(tf.keras.layers.Conv1D(filters=64, kernel_size=3,
                      strides=1,
                      activation="relu",
                      padding='causal',))
    model.add(tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(64,input_shape=input_shape,return_sequences=True)))
    model.add(tf.keras.layers.LSTM(32))
    model.add(tf.keras.layers.Dense(16))
    model.add(tf.keras.layers.Dropout(0.2))
    model.add(tf.keras.layers.Dense(6))
    model.add(tf.keras.layers.Lambda(lambda x: x*norm.mean))
    model.add(tf.keras.layers.Dense(1,activation='linear'))
    model.summary()
    return model

model = get_uncompile_model(SERIES_SHAPE,normalizer_layer)
history = compile_fit_model(model,3,train_series,val_series,1e-8)

问题描述 #

目前验证集预测结果完全不在合理范围内，即使未调参也应落在合理区间。尝试过修改输出层线性激活函数、调整网络结构，但均无效果。参考TensorFlow入门回归教程后仍未找到问题。

新增Dropout、Dense(6)和Lambda层后，训练10轮的验证集预测结果如下：

排查建议 #

• 修正归一化逆变换逻辑：当前模型中Lambda(lambda x: x*norm.mean)的逆变换错误，Normalization层的正确逆变换应为x * norm.variance**0.5 + norm.mean，仅乘均值会导致数值缩放完全偏离真实范围，这很可能是核心问题。
• 调整学习率：当前设置的1e-8学习率过小，模型几乎无法更新参数，尝试将学习率调整到1e-4或1e-3后重新训练。
• 验证数据集处理一致性：确保验证集使用的是训练集适配好的归一化层，不能单独对验证集适配Normalization层，否则会导致数据分布不匹配。
• 检查窗口生成逻辑：手动抽取几个窗口样本，确认特征窗口（window[:-window_jump]）和目标值（window[-target:][:,dim_target]）的对应关系是否正确，避免出现时间步错位。
• 验证模型训练状态：查看训练时的loss曲线，如果loss始终不下降，说明模型未有效学习，可先简化网络结构（比如去掉Conv1D和Bidirectional LSTM），验证基础模型是否能正常拟合。
• 临时移除逆变换层：先去掉Lambda层，让模型在归一化后的数值上训练，观察预测结果是否在归一化后的合理区间（如[-3,3]），再添加正确的逆变换。

内容的提问来源于stack exchange，提问作者Andrés Tello Urrea