解决TensorFlow中InceptionV3-LSTM模型的InvalidArgumentError图执行错误

问题描述 #

尝试训练以InceptionV3为编码器、LSTM为解码器的编解码器模型时，出现以下GPU兼容性错误：

InvalidArgumentError: Graph execution error: No DNN in stream executor.

已尝试强制CPU运行，但问题仍存在。原训练代码如下：

# Use InceptionV3 as a Pre-Trained Model for Feature Extraction
from tensorflow.keras.applications import InceptionV3
from tensorflow.keras.applications.inception_v3 import preprocess_input
from tensorflow.keras.layers import LSTM, TimeDistributed, Dense, Input, GlobalAveragePooling2D, RepeatVector
import tensorflow as tf

print("Using InceptionV3 for feature extraction...")
base_model = InceptionV3(weights='imagenet', include_top=False)
for layer in base_model.layers:
    layer.trainable = False

# Define encoder
input_layer = Input(shape=(224, 224, 3), dtype='float32')
preprocessed = preprocess_input(input_layer)
features = base_model(preprocessed)
flat_features = GlobalAveragePooling2D()(features)
encoded_features = Dense(256, activation='relu')(flat_features)  # Example size

# Define LSTM decoder
decoder_input = Input(shape=(256,), dtype='float32')
repeated_layer = RepeatVector(20)(decoder_input)  # Example size
lstm_1 = LSTM(512, return_sequences=True, dropout=0.3)(repeated_layer)
output_layer = TimeDistributed(Dense(1000, activation='softmax'))(lstm_1)

# Compile the Encoder-Decoder Model
model_input = Input(shape=(224, 224, 3), dtype='float32')
encoded_features = Dense(256, activation='relu')(flat_features)
decoded_output = TimeDistributed(Dense(1000, activation='softmax'))(encoded_features)
encoder_decoder_model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])

# Train the Model
encoder_decoder_model.fit(x_train, y_train, validation_data=(x_test, y_test), batch_size=16, epochs=5)

# Save the Model
encoder_decoder_model.save('Trained_Model_with_InceptionV3_and_GloVe.h5')

解决方案 #

1. 修复代码中的模型定义错误 #

原代码存在多处逻辑错误，这可能是触发错误的间接原因：

• 未用Model类实例化encoder_decoder_model，解码器与编码器未正确连接
• 重复定义encoded_features，覆盖了编码器输出
• 解码器未调用LSTM层，直接用TimeDistributed处理特征

修正后的代码：

from tensorflow.keras.applications import InceptionV3
from tensorflow.keras.applications.inception_v3 import preprocess_input
from tensorflow.keras.layers import LSTM, TimeDistributed, Dense, Input, GlobalAveragePooling2D, RepeatVector
from tensorflow.keras.models import Model  # 新增：导入Model类
import tensorflow as tf

print("Using InceptionV3 for feature extraction...")
base_model = InceptionV3(weights='imagenet', include_top=False)
for layer in base_model.layers:
    layer.trainable = False

# 定义编码器
input_layer = Input(shape=(224, 224, 3), dtype='float32')
preprocessed = preprocess_input(input_layer)
features = base_model(preprocessed)
flat_features = GlobalAveragePooling2D()(features)
encoded_features = Dense(256, activation='relu')(flat_features)

# 定义解码器（与编码器直接连接）
repeated_layer = RepeatVector(20)(encoded_features)
lstm_1 = LSTM(512, return_sequences=True, dropout=0.3)(repeated_layer)
output_layer = TimeDistributed(Dense(1000, activation='softmax'))(lstm_1)

# 实例化编解码器模型
encoder_decoder_model = Model(inputs=input_layer, outputs=output_layer)

# 编译模型
encoder_decoder_model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])

# 训练模型（确保x_train、y_train已完成预处理）
encoder_decoder_model.fit(x_train, y_train, validation_data=(x_test, y_test), batch_size=16, epochs=5)

# 保存模型
encoder_decoder_model.save('Trained_Model_with_InceptionV3_and_LSTM.h5')

2. 处理GPU/CPU兼容性问题 #

针对GPU运行的修复

• 匹配TensorFlow与CUDA/CUDNN版本：确保安装的TensorFlow版本和CUDA、CUDNN版本完全兼容（例如TensorFlow 2.15对应CUDA 12.2、CUDNN 8.9）
• 指定单GPU运行：多GPU环境下，可限制仅使用一块GPU：

gpus = tf.config.list_physical_devices('GPU')
if gpus:
    try:
        tf.config.set_visible_devices(gpus[0], 'GPU')
        logical_gpus = tf.config.list_logical_devices('GPU')
        print(len(gpus), "Physical GPUs,", len(logical_gpus), "Logical GPU")
    except RuntimeError as e:
        print(e)

• 禁用DNN加速优化：若GPU不支持部分DNN操作，可关闭相关优化：

tf.config.optimizer.set_jit(False)

针对CPU运行的修复

• 彻底禁用GPU：在代码最开头添加以下代码，强制TensorFlow仅使用CPU：

import os
os.environ["CUDA_VISIBLE_DEVICES"] = "-1"
import tensorflow as tf

• 检查CPU指令集兼容性：老旧CPU若不支持AVX2等指令集，需安装适配的TensorFlow版本（如tf-nightly-cpu）

3. 其他排查步骤 #

• 运行print(tf.config.list_physical_devices('GPU'))，确认TensorFlow是否识别到GPU，若输出为空需重新安装显卡驱动和CUDA/CUDNN
• 降低batch size：将batch_size从16改为8或4，避免显存不足引发的错误
• 更新显卡驱动：确保显卡驱动为最新版本，适配当前CUDA版本

内容的提问来源于stack exchange，提问作者Buragadda Manikanta Lokesh