如何在Keras构建的LSTM神经网络中使用SHAP计算Shapley值？

问题背景 #

使用Keras构建LSTM模型时，LSTM要求输入为三维格式（样本数，时间步，特征数），但SHAP的KernelExplainer默认适配二维输入，直接转换维度会导致模型或SHAP报错。以下是复现该问题的代码：

import numpy as np
from random import uniform

N=100

#Initlaize input/output vectors
x1=[] 
x2=[] 
x3=[]
y1=[]
y2=[]

#Generate some data
for i in range(N):
    x1.append(i/100+uniform(-.1,.1))
    x2.append(i/100+uniform(-3,5)+2)
    x3.append(uniform(0,1)/np.sqrt(i+1))
    
    y1.append(2*x1[i]-.5*x2[i]+x3[i]+uniform(-1,1))
    y2.append(x1[i]+3*x3[i]+5+uniform(-1,3))

#Convert lists to numpy arrays
x1=np.array(x1).reshape(N,1)
x2=np.array(x2).reshape(N,1)
x3=np.array(x3).reshape(N,1)

y1=np.array(y1).reshape(N,1)

#Assemble into matrices
X = np.hstack((x1, x2, x3))
Y = y1

# reshape input to be [samples, time steps, features]
X = np.reshape(X, (X.shape[0], 1, X.shape[1]))

#Import keras functions
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import LSTM


#Lets build us a neural net!
model=Sequential()
model.add(LSTM(4, input_shape=(1,3)))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='adam',run_eagerly=())
model.fit(X, Y, epochs=100, batch_size=10, verbose=2)


import shap
import tensorflow as tf
tf.compat.v1.disable_eager_execution()

DE=shap.KernelExplainer(model.predict,shap.sample(X,10))
shap_values = DE.shap_values(X) # X is 3d numpy.ndarray

解决思路 #

核心是通过包装预测函数统一模型与SHAP的输入维度要求：让SHAP处理原始二维数据，在预测函数内部完成向LSTM所需三维格式的转换，避免直接修改数据维度引发的冲突。

修改后的完整代码 #

import numpy as np
from random import uniform
import shap
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, LSTM

# 生成数据
N = 100
x1 = [] 
x2 = [] 
x3 = []
y1 = []

for i in range(N):
    x1.append(i/100 + uniform(-.1, .1))
    x2.append(i/100 + uniform(-3, 5) + 2)
    x3.append(uniform(0, 1)/np.sqrt(i+1))
    y1.append(2*x1[i] - .5*x2[i] + x3[i] + uniform(-1, 1))

# 转换为数组并组装为二维格式（供SHAP使用）
x1 = np.array(x1).reshape(N, 1)
x2 = np.array(x2).reshape(N, 1)
x3 = np.array(x3).reshape(N, 1)
y1 = np.array(y1).reshape(N, 1)

X = np.hstack((x1, x2, x3))
Y = y1

# 构建LSTM模型，保持原输入形状要求
model = Sequential()
model.add(LSTM(4, input_shape=(1, 3)))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='adam')
# 训练时临时转换为三维输入
model.fit(np.reshape(X, (X.shape[0], 1, X.shape[1])), Y, epochs=100, batch_size=10, verbose=2)

# 包装预测函数：适配SHAP的二维输入，内部转为LSTM需要的三维格式
def lstm_predict(x):
    x_3d = np.reshape(x, (x.shape[0], 1, x.shape[1]))
    return model.predict(x_3d, verbose=0)

# 初始化SHAP解释器，传入二维数据样本
explainer = shap.KernelExplainer(lstm_predict, shap.sample(X, 10))
# 计算Shapley值，直接传入二维的X
shap_values = explainer.shap_values(X)

# 可选：可视化特征重要性
shap.summary_plot(shap_values, X, feature_names=["x1", "x2", "x3"])

关键修改说明 #

• 预测函数包装：lstm_predict函数作为中间层，接收SHAP传入的二维数据，转换为LSTM要求的三维格式后再调用模型预测，完美解决维度冲突
• 数据维度分离：训练模型时临时转换维度，SHAP全程使用原始二维数据，避免重复转换导致的逻辑混乱
• 清理冗余配置：移除了run_eagerly=()和tf.compat.v1.disable_eager_execution()，避免TensorFlow执行模式冲突

内容的提问来源于stack exchange，提问作者makkel