调用scaler.inverse_transform()时出现维度不匹配ValueError问题求助

问题现象 #

基于LSTM的股票价格预测代码运行正常，但执行predicted_stock_price = scaler.inverse_transform(predicted_stock_price)时触发错误：

ValueError: Found array with dim 3. Estimator expected <= 2.

完整错误栈：

ValueError                                Traceback (most recent call last)
<ipython-input-224-dacdc53f9aa9> in <module>
      1 #PREDIZIONE
      2 predicted_stock_price = model1.predict(X_test)
----> 3 predicted_stock_price = scaler.inverse_transform(predicted_stock_price)
      4 
      5 #nsamples, nx, ny = predicted_stock_price.shape

1 frames
/usr/local/lib/python3.7/dist-packages/sklearn/preprocessing/_data.py in inverse_transform(self, X)
    524 
    525         X = check_array(
--> 526             X, copy=self.copy, dtype=FLOAT_DTYPES, force_all_finite="allow-nan"
    527         )
    528 

/usr/local/lib/python3.7/dist-packages/sklearn/utils/validation.py in check_array(array, accept_sparse, accept_large_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, ensure_min_samples, ensure_min_features, estimator)
    794             raise ValueError(
    795                 "Found array with dim %d. %s expected <= 2."
--> 796                 % (array.ndim, estimator_name)
    797             )
    798 

ValueError: Found array with dim 3. Estimator expected <= 2.

错误原因 #

你的LSTM模型中所有LSTM层都设置了return_sequences=True，这个参数会让LSTM层返回每个时间步的输出，导致最终模型的预测结果是3维数组（形状为(样本数, 时间步长, 1)）。而MinMaxScaler是基于2维数据（(样本数, 特征数)）拟合的，inverse_transform方法只接受2维及以下的数组，因此触发维度不匹配错误。

此外，股票价格预测任务中，我们只需要每个输入序列对应一个预测值，不需要每个时间步都输出结果，所以最后一个LSTM层设置return_sequences=True是不合理的。

解决方案 #

方案1：修改模型结构（推荐） #

将最后一个LSTM层的return_sequences改为False，这样最后一个LSTM层会返回每个样本的最终输出（2维数组），后续的Dense层输出也会是2维，完全匹配scaler.inverse_transform的要求。

修改后的模型结构代码：

#LSTM
model1 = Sequential()

model1.add(LSTM(70, activation = "relu", return_sequences = True, input_shape = (X_allenamento.shape[1], 1)))
model1.add(Dropout(0.2))

model1.add(LSTM(90, activation = "relu", return_sequences = True))
model1.add(Dropout(0.3))

model1.add(LSTM(120, activation = "relu", return_sequences = True))
model1.add(Dropout(0.4))

# 修改最后一个LSTM层的return_sequences为False
model1.add(LSTM(150, activation = "relu", return_sequences = False))
model1.add(Dropout(0.6))

model1.add(Dense(1))

方案2：临时降维（不推荐，仅用于无法修改模型的场景） #

如果必须保留原模型结构，可以在逆转换前将3维的预测结果降为2维。取每个样本最后一个时间步的输出即可：

predicted_stock_price = model1.predict(X_test)
# 取每个样本最后一个时间步的预测值，将形状从(样本数,60,1)转为(样本数,1)
predicted_stock_price = predicted_stock_price[:, -1, :]
predicted_stock_price = scaler.inverse_transform(predicted_stock_price)

完整修正代码 #

import pandas_datareader as web
from datetime import datetime
from sklearn.preprocessing import MinMaxScaler
import pandas as pd
import numpy as np
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense, LSTM, Dropout

ticker = "AAPL"

while True:
    data_inizio = str(input("data d'inizi dell'analisi (inserisci giorno mese ed anno): " ))
    try:
        start1 = datetime.strptime(data_inizio, '%d %m %Y')
        break
    except ValueError:
        print('il formato inserito non è corretto scrivere la data inserendo giorno mese ed anno separati dallo spazio es. 20 01 2021')

while True:
    data_fine = str(input("data di fine analisi (inserisci giorno mese ed anno): " ))
    try:
        end1 = datetime.strptime(data_f_fine, '%d %m %Y')
        break
    except ValueError:
        print('il formato inserito non è corretto scrivere la data inserendo giorno mese ed anno separati dallo spazio es. 20 01 2021')

inizio_analisi = start1
fine_analisi = end1


dati_finanziari_ticker = web.DataReader(ticker, data_source="yahoo", start=inizio_analisi, end=fine_analisi)
periodo_analisi = dati_finanziari_ticker.shape[0]
periodo_allenamento_programma = round(periodo_analisi*0.8)
periodo_test_programma = periodo_analisi
dati_finanziari_ticker_scalati = dati_finanziari_ticker.iloc[:, 3:4].values


scaler = MinMaxScaler(feature_range=(0,1))
dati_allenamento_scalati = scaler.fit_transform(dati_finanziari_ticker_scalati)


X_allenamento = [] 
y_allenamento = []

for i in range(60, periodo_allenamento_programma):
  X_allenamento.append(dati_finanziari_ticker_scalati[i-60:i, 0])
  y_allenamento.append(dati_finanziari_ticker_scalati[i,0])


X_allenamento, y_allenamento = np.array(X_allenamento), np.array(y_allenamento)
X_allenamento = np.reshape(X_allenamento, (X_allenamento.shape[0], X_allenamento.shape[1], 1))

#LSTM
model1 = Sequential()

model1.add(LSTM(70, activation = "relu", return_sequences = True, input_shape = (X_allenamento.shape[1], 1)))
model1.add(Dropout(0.2))

model1.add(LSTM(90, activation = "relu", return_sequences = True))
model1.add(Dropout(0.3))

model1.add(LSTM(120, activation = "relu", return_sequences = True))
model1.add(Dropout(0.4))

# 修改此处的return_sequences为False
model1.add(LSTM(150, activation = "relu", return_sequences = False))
model1.add(Dropout(0.6))

model1.add(Dense(1))

model1.summary()  # 修正：添加括号以正确打印模型摘要

model1.compile(optimizer = "adam", loss = "mean_squared_error")

model1.fit(X_allenamento, y_allenamento, epochs=10, batch_size=28)

#CREAZIONE DEGLI ARRAY PER IL TEST
X_test = [] 
y_test = []

for i in range(periodo_allenamento_programma, periodo_test_programma):
  X_test.append(dati_finanziari_ticker_scalati[i-60:i, 0])
  y_test.append(dati_finanziari_ticker_scalati[i,0])


X_test, y_test = np.array(X_test), np.array(y_test)
X_test = np.reshape(X_test, (X_test.shape[0], X_test.shape[1], 1))


predicted_stock_price = model1.predict(X_test)
predicted_stock_price = scaler.inverse_transform(predicted_stock_price)

内容的提问来源于stack exchange，提问作者Francesco Stocchi