DL4J搭建LSTM回归模型：测试输出值均异常一致问题求助

安全阿岳

2026-8-2

问题：DeepLearning4J LSTM回归模型输出无明显差异

我使用DeepLearning4J搭建了一个输入特征数为2、时间序列长度为1的LSTM回归模型，但训练完成后，输入不同测试数据得到的输出值没有明显差异。已尝试调整优化器、隐藏层激活函数、学习率等超参数，问题仍未解决。

实现代码

public class LSTMRegression {
    public static final int inputSize = 2,
                            lstmLayerSize = 4,
                            outputSize = 1;
    
    public static final double learningRate = 0.0001;

    public static void main(String[] args) {
        int miniBatchSize = 99;
        
        MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder()
                .miniBatch(false)
                .optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)
                .updater(new Adam(learningRate))
                .list()
                .layer(0, new LSTM.Builder().nIn(inputSize).nOut(lstmLayerSize)
                        .weightInit(WeightInit.XAVIER)
                        .activation(Activation.TANH).build())
//                .layer(1, new LSTM.Builder().nIn(lstmLayerSize).nOut(lstmLayerSize)
//                        .weightInit(WeightInit.XAVIER)
//                        .activation(Activation.SIGMOID).build())
//                .layer(2, new LSTM.Builder().nIn(lstmLayerSize).nOut(lstmLayerSize)
//                        .weightInit(WeightInit.XAVIER)
//                        .activation(Activation.SIGMOID).build())
                .layer(1, new RnnOutputLayer.Builder(LossFunctions.LossFunction.MSE)
                        .weightInit(WeightInit.XAVIER)
                        .activation(Activation.IDENTITY)
                        .nIn(lstmLayerSize).nOut(outputSize).build())
                
                .backpropType(BackpropType.TruncatedBPTT)
                .tBPTTForwardLength(miniBatchSize)
                .tBPTTBackwardLength(miniBatchSize)
                .build();
        
        final var network = new MultiLayerNetwork(conf);
        final DataSet train = getTrain();
        final INDArray test = getTest();
        
        final DataNormalization normalizer = new NormalizerMinMaxScaler(0, 1);
//                                          = new NormalizerStandardize();
        
        normalizer.fitLabel(true);
        normalizer.fit(train);

        normalizer.transform(train);
        normalizer.transform(test);
        
        network.init();
        
        for (int i = 0; i < 100; i++)
            network.fit(train);
        
        final INDArray output = network.output(test);
        
        normalizer.revertLabels(output);
        
        System.out.println(output);
    }
    
    public static INDArray getTest() {
        double[][][] test = new double[][][]{
            {{20}, {203}},
            {{16}, {183}},
            {{20}, {190}},
            {{18.6}, {193}},
            {{18.9}, {184}},
            {{17.2}, {199}},
            {{20}, {190}},
            {{17}, {181}},
            {{19}, {197}},
            {{16.5}, {198}},
            ...
        };
        
        INDArray input = Nd4j.create(test);
        
        return input;
    }
    
    public static DataSet getTrain() {
        double[][][] inputArray = {
            {{18.7}, {181}},
            {{17.4}, {186}},
            {{18}, {195}},
            {{19.3}, {193}},
            {{20.6}, {190}},
            {{17.8}, {181}},
            {{19.6}, {195}},
            {{18.1}, {193}},
            {{20.2}, {190}},
            {{17.1}, {186}},
            ...
        };
        
        double[][] outputArray = {
                {3750},
                {3800},
                {3250},
                {3450},
                {3650},
                {3625},
                {4675},
                {3475},
                {4250},
                {3300},
                ...
        };
        
        INDArray input = Nd4j.create(inputArray);
        INDArray labels = Nd4j.create(outputArray);
        
        return new DataSet(input, labels);
    }
}

运行警告与输出

00:06:04.554 [main] WARN  o.d.nn.multilayer.MultiLayerNetwork - Cannot do truncated BPTT with non-3d inputs or labels. Expect input with shape [miniBatchSize,nIn,timeSeriesLength], got [99, 2, 1] and labels with shape [99, 1]
00:06:04.554 [main] WARN  o.d.nn.multilayer.MultiLayerNetwork - Cannot do truncated BPTT with non-3d inputs or labels. Expect input with shape [miniBatchSize,nIn,timeSeriesLength], got [99, 2, 1] and labels with shape [99, 1]
00:06:04.555 [main] WARN  o.d.nn.multilayer.MultiLayerNetwork - Cannot do truncated BPTT with non-3d inputs or labels. Expect input with shape [miniBatchSize,nIn,timeSeriesLength], got [99, 2, 1] and labels with shape [99, 1]
00:06:04.555 [main] WARN  o.d.nn.multilayer.MultiLayerNetwork - Cannot do truncated BPTT with non-3d inputs or labels. Expect input with shape [miniBatchSize,nIn,timeSeriesLength], got [99, 2, 1] and labels with shape [99, 1]
00:06:04.555 [main] WARN  o.d.nn.multilayer.MultiLayerNetwork - Cannot do truncated BPTT with non-3d inputs or labels. Expect input with shape [miniBatchSize,nIn,timeSeriesLength], got [99, 2, 1] and labels with shape [99, 1]
00:06:04.555 [main] WARN  o.d.nn.multilayer.MultiLayerNetwork - Cannot do truncated BPTT with non-3d inputs or labels. Expect input with shape [miniBatchSize,nIn,timeSeriesLength], got [99, 2, 1] and labels with shape [99, 1]
00:06:04.555 [main] WARN  o.d.nn.multilayer.MultiLayerNetwork - Cannot do truncated BPTT with non-3d inputs or labels. Expect input with shape [miniBatchSize,nIn,timeSeriesLength], got [99, 2, 1] and labels with shape [99, 1]
00:06:04.555 [main] WARN  o.d.nn.multilayer.MultiLayerNetwork - Cannot do truncated BPTT with non-3d inputs or labels. Expect input with shape [miniBatchSize,nIn,timeSeriesLength], got [99, 2, 1] and labels with shape [99, 1]
00:06:04.555 [main] WARN  o.d.nn.multilayer.MultiLayerNetwork - Cannot do truncated BPTT with non-3d inputs or labels. Expect input with shape [miniBatchSize,nIn,timeSeriesLength], got [99, 2, 1] and labels with shape [99, 1]
00:06:04.555 [main] WARN  o.d.nn.multilayer.MultiLayerNetwork - Cannot do truncated BPTT with non-3d inputs or labels. Expect input with shape [miniBatchSize,nIn,timeSeriesLength], got [99, 2, 1] and labels with shape [99, 1]

[[[3198.1614]], 

 [[2986.7781]], 

 [[3059.7017]], 

 [[3105.3828]], 

 [[2994.0127]], 

 [[3191.4468]], 

 [[3059.7017]], 

 [[2962.4341]], 

 [[3147.4412]], 

 [[3183.5991]]]

解决方案

1. 修复Truncated BPTT配置问题

警告明确指出无法使用Truncated BPTT，因为标签是2D格式而非要求的3D。由于你的时间序列长度为1，完全不需要使用Truncated BPTT，改用普通反向传播即可：

MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder()
        // 保留其他配置
        .backpropType(BackpropType.Standard)
        // 移除.tBPTTForwardLength和.tBPTTBackwardLength两行配置
        .build();

2. 调整训练策略

提高学习率：当前0.0001的学习率过低，模型收敛速度极慢，尝试调整为0.001或0.01，同时监控训练损失变化。
增加训练轮次：100轮不足以让模型充分学习数据模式，增加到500-1000轮，并打印损失观察收敛情况：

for (int i = 0; i < 1000; i++) {
    double loss = network.fit(train);
    if (i % 100 == 0) {
        System.out.println("第" + i + "轮训练损失：" + loss);
    }
}

调整批量大小：当前批量大小等于训练样本总数，属于批量梯度下降，尝试改为16或32的小批量，帮助模型更好地捕捉数据规律。

3. 优化模型结构

增加LSTM神经元数量：当前LSTM层仅4个神经元，模型容量不足，尝试提升到16或32个。
添加全连接层增强表达：在LSTM层和输出层之间加入全连接层，提升模型拟合能力：

.layer(0, new LSTM.Builder().nIn(inputSize).nOut(16)
        .weightInit(WeightInit.XAVIER)
        .activation(Activation.TANH).build())
.layer(1, new DenseLayer.Builder().nIn(16).nOut(8)
        .weightInit(WeightInit.XAVIER)
        .activation(Activation.RELU).build())
.layer(2, new RnnOutputLayer.Builder(LossFunctions.LossFunction.MSE)
        .weightInit(WeightInit.XAVIER)
        .activation(Activation.IDENTITY)
        .nIn(8).nOut(outputSize).build())

4. 检查数据归一化

尝试替换归一化器为NormalizerStandardize，部分场景下标准化比MinMax缩放更有利于模型收敛：

final DataNormalization normalizer = new NormalizerStandardize();
normalizer.fitLabel(true);
normalizer.fit(train);
normalizer.transform(train);
normalizer.transform(test);

5. 考虑替换模型类型

由于你的时间序列长度为1，LSTM的时序记忆优势无法发挥，完全可以用普通全连接神经网络替代，可能会得到更好的效果：

MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder()
        .miniBatch(false)
        .optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)
        .updater(new Adam(0.001))
        .list()
        .layer(0, new DenseLayer.Builder().nIn(inputSize).nOut(16)
                .weightInit(WeightInit.XAVIER)
                .activation(Activation.RELU).build())
        .layer(1, new DenseLayer.Builder().nIn(16).nOut(8)
                .weightInit(WeightInit.XAVIER)
                .activation(Activation.RELU).build())
        .layer(2, new OutputLayer.Builder(LossFunctions.LossFunction.MSE)
                .weightInit(WeightInit.XAVIER)
                .activation(Activation.IDENTITY)
                .nIn(8).nOut(outputSize).build())
        .build();

内容的提问来源于stack exchange，提问作者Twisted Tea

DL4J搭建LSTM回归模型：测试输出值均异常一致问题求助

实现代码 #

运行警告与输出 #

解决方案 #

1. 修复Truncated BPTT配置问题 #

2. 调整训练策略 #

3. 优化模型结构 #

4. 检查数据归一化 #

5. 考虑替换模型类型 #