TensorFlow中多组非拼接时间序列训练单神经网络的有效性验证

我想用TensorFlow构建一个工程系统的代理神经网络，输入是1个信号（比如阶跃输入），预测1个输出（比如位移）。我有多组长度不同的时间序列数据，每组对应系统的不同工况，必须让网络学习所有场景。这些数据来自不同实验的连续时间序列，无法拼接成单个数据集。

目前我的做法是：先构建模型，再循环读取每个数据集，每次调用model.fit()训练。我查到资料说连续调用model.fit()不会清除之前的训练，而是累加训练，想问这是否正确？另外，这种训练方式能不能保证网络学到真实系统的所有行为？

我的训练代码如下：

from pathlib import Path
import pandas as pd
import datetime
import sklearn
import numpy as np
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import LSTM, Dense
from tensorflow.keras.optimizers import RMSprop
from tensorflow.keras.callbacks import ModelCheckpoint, EarlyStopping, ReduceLROnPlateau
from sklearn.preprocessing import MinMaxScaler
from tensorflow.keras.regularizers import L1L2

directory = Path(r'C:\Users\User\Documents\Mass_Spring_Damper') # Raw training files directory
column_names = ['Time', 'Displacement', 'Step input'] # Time column, output column, input column

# Create model
model = Sequential()

model.add(LSTM(units=200,
                    activation='relu',
                    return_sequences=True,
                    kernel_regularizer=L1L2(l1=0.01, l2=0.0),
                    input_shape=(None, 1)))

model.add(Dense(1,
                activation='linear'))

# Compile model with optimizer
optimizer = RMSprop(learning_rate=1e-3, clipvalue=1)

model.compile(loss='mean_squared_error', optimizer=optimizer)

model.summary()

# Train model over multiple datasets in loop
for file in directory.glob('*.csv'):

    dataset = pd.read_csv(file, index_col=False, names=column_names)

    # Parse time column as datetime
    s = dataset['Time']
    start = datetime.datetime(2023, 1, 1, 0, 0)  # Reference date, choose any!
    datetime_series = s.astype('timedelta64[s]') + start
    time_series = datetime_series.dt.time
    dataset = dataset.drop(['Time'], axis=1)
    dataset.insert(0, 'Time', time_series, True)  # Insert datetime column into raw dataset
    dataset['Time'] = pd.to_datetime(dataset['Time'], format='%H:%M:%S')
    dataset = dataset.set_index('Time')

    # Drop NaN values
    dataset = dataset.dropna()

    # Train-test-validation split
    train_dataset, test_dataset = sklearn.model_selection.train_test_split(dataset,
                                                                           test_size=0.1,
                                                                           shuffle=False,
                                                                           random_state=42)

    train_dataset, val_dataset = sklearn.model_selection.train_test_split(train_dataset,
                                                                          test_size=0.2,
                                                                          shuffle=False,
                                                                          random_state=42)


    # Define input and target signals
    input_signals = ['Step input']
    target_signals = ['Displacement']

    # Get x_train, y_train, x_test, y_test, x_val, y_val
    train_dataset_input = train_dataset[input_signals]
    x_train = train_dataset_input.values

    train_dataset_target = train_dataset[target_signals]
    y_train = train_dataset_target.values

    test_dataset_input = test_dataset[input_signals]
    x_test = test_dataset_input.values

    test_dataset_target = test_dataset[target_signals]
    y_test = test_dataset_target.values

    val_dataset_input = val_dataset[input_signals]
    x_val = val_dataset_input.values

    val_dataset_target = val_dataset[target_signals]
    y_val = val_dataset_target.values

    # Find and replace invalid data in arrays
    x_train[x_train < 1e-9] = 0
    y_train[y_train < 1e-9] = 0
    x_test[x_test < 1e-9] = 0
    y_test[y_test < 1e-9] = 0
    x_val[x_val < 1e-9] = 0
    y_val[y_val < 1e-9] = 0

    train_size = len(x_train)  # Size of training set (used later for input generator)
    num_x_signals = x_train.shape[1]  # Number of input signals
    num_y_signals = y_train.shape[1]  # Number of output signals

    # Scale data
    x_scaler = MinMaxScaler((0, 1))
    x_train_scaled = x_scaler.fit_transform(x_train)
    x_test_scaled = x_scaler.transform(x_test)
    x_val_scaled = x_scaler.transform(x_val)

    y_scaler = MinMaxScaler((0, 1))
    y_train_scaled = y_scaler.fit_transform(y_train)
    y_test_scaled = y_scaler.transform(y_test)
    y_val_scaled = y_scaler.transform(y_val)

    # Random training batch generator 
    def batch_generator(batch_size, sequence_length):
        
        while True:
            # Allocate a new array for the batch of input-signals.
            x_shape = (batch_size, sequence_length, num_x_signals)
            x_batch = np.zeros(shape=x_shape, dtype=np.float16)

            # Allocate a new array for the batch of output-signals.
            y_shape = (batch_size, sequence_length, num_y_signals)
            y_batch = np.zeros(shape=y_shape, dtype=np.float16)

            # Fill the batch with random sequences of data.
            for i in range(batch_size):
                # Get a random start-index.
                # This points somewhere into the training-data.
                idx = np.random.randint(train_size - sequence_length)

                # Copy the sequences of data starting at this index.
                x_batch[i] = x_train_scaled[idx:idx + sequence_length]
                y_batch[i] = y_train_scaled[idx:idx + sequence_length]

            yield (x_batch, y_batch)

    batch_size = 128
    sequence_length = 50
    generator = batch_generator(batch_size=batch_size, sequence_length=sequence_length)

    # Callbacks
    path_checkpoint = '23_checkpoint.keras'
    callback_checkpoint = ModelCheckpoint(filepath=path_checkpoint,
                                          monitor='val_loss',
                                          verbose=1,
                                          save_weights_only=True,
                                          save_best_only=True)  # Callback for writing checkpoints during training

    callback_early_stopping = EarlyStopping(monitor='val_loss',
                                            patience=5,
                                            verbose=1)  # Callback for stopping optimization when performance worsens on validation set

    callback_reduce_lr = ReduceLROnPlateau(monitor='val_loss',
                                           factor=0.1,
                                           min_lr=1e-5,
                                           patience=0,
                                           verbose=1)  # Callback for reducing learning rate when validation loss does not improve

    callbacks = [callback_checkpoint, callback_reduce_lr, callback_early_stopping]

    # Define validation data
    validation_data = (np.expand_dims(x_val_scaled, axis=0), np.expand_dims(y_val_scaled, axis=0))

    # Train model
    history = model.fit(x=generator,
                        epochs=100,
                        validation_data=validation_data,
                        callbacks=callbacks,
                        steps_per_epoch=50)

1. 连续调用model.fit()是否会累加训练？ #

这个说法是正确的。在TensorFlow/Keras中，同一个模型实例多次调用model.fit()时，会基于之前训练得到的权重继续更新，不会重置权重（除非手动调用model.reset_states()或重新初始化模型）。每次fit()都会延续之前的优化状态（比如优化器的动量、学习率调度等），相当于把多个数据集的训练过程串联起来。

2. 这种训练方式能否让网络学习到系统的所有行为？ #

不一定，取决于训练过程的合理性。这种“逐数据集循环训练”的思路本身可行，但如果细节处理不当，很容易出现模型偏向最后训练的数据集（遗忘之前的工况），或者对某些工况学习不足的问题。要让模型学到所有工况，需要解决数据分布差异、训练稳定性等问题，你的代码存在一些需要优化的点。

• 数据归一化标准不一致：你在每个数据集循环中重新初始化MinMaxScaler并拟合当前训练数据，导致不同数据集的归一化规则不同。模型后续训练时，输入输出的缩放逻辑变化，会干扰权重更新，甚至抵消之前的训练效果。
  优化：用所有数据集的联合统计量拟合scaler；若数据集范围差异不大，可在第一个数据集上拟合后，后续所有数据集都用同一scaler做转换；若差异大，考虑用标准化（StandardScaler），或为每个数据集单独保存scaler，预测时对应使用。

• 验证集目标漂移：每个数据集都拆分独立验证集，并用它触发早停、学习率调整和权重保存。这会导致模型的验证目标不断切换（比如先监控工况A的验证loss，再切换到工况B），早停可能在工况B的loss下降时停止，但此时模型对工况A的性能可能已经下降。
  优化：准备一个跨工况的独立验证集（包含所有工况的样本），每次训练都用这个统一验证集监控性能，避免验证目标漂移。

• 早停与权重保存逻辑不合理：当前早停会在单个数据集的验证loss不下降时终止训练，导致后续数据集无法被学习；且ModelCheckpoint每次都会覆盖之前的 checkpoint，最终只保留最后一个数据集训练时的最优权重，丢失之前的训练成果。
  优化：要么去掉单数据集的早停，改为固定训练轮数；要么调整早停监控指标为统一验证集的loss，同时修改ModelCheckpoint的保存策略（比如按数据集保存，或只保存全局验证loss最优的权重）。

• 生成器易导致工况偏向：每个数据集的生成器仅从当前数据集采样序列，容易让模型短期内过度拟合当前工况，加剧对之前工况的遗忘。
  优化：实现多数据集混合生成器，每次从所有已加载的数据集中随机采样序列，让模型同时接触不同工况的数据，缓解遗忘问题。

• LSTM激活函数选择不当：你用relu作为LSTM的激活函数，而LSTM内部默认使用tanh。relu在循环层中容易引发梯度消失问题，建议改用tanh或swish。

内容的提问来源于stack exchange，提问作者glafira_d