TensorFlow训练随机卡顿并报设备拷贝警告,如何保留eager execution解决?
问题描述
我设置了以下代码:
tf.config.experimental.set_device_policy('warn')
调用fit()函数时,训练过程会在随机步骤和轮次出现卡顿,且每一步都会触发如下警告:
W tensorflow/core/common_runtime/eager/execute.cc:169] before computing Shape input #0 was expected to be on /job:localhost/replica:0/task:0/device:GPU:0 but is actually on /job:localhost/replica:0/task:0/device:CPU:0 (operation running on /job:localhost/replica:0/task:0/device:GPU:0). This triggers a copy which can be a performance bottleneck.
尝试用tf.compat.v1.disable_eager_execution()解决问题,起初似乎有效,但该方法会禁用我需要的部分功能,后续发现这个方法也不再起作用。
附完整代码
import tensorflow as tf from tensorflow import keras from keras import layers, callbacks path = 'Dogs/' train_data = keras.utils.image_dataset_from_directory(path, subset= 'training', validation_split= 0.2, seed= 478, label_mode= 'categorical', batch_size= 32) test_data = keras.utils.image_dataset_from_directory(path, subset= 'validation', validation_split= 0.2, seed= 478, label_mode= 'categorical', batch_size= 32) def standartize_img(image, label): image = image / 255.0 return image, label AUTOTUNE = tf.data.AUTOTUNE train_data = train_data.map(standartize_img).cache().prefetch(AUTOTUNE) test_data = test_data.map(standartize_img).cache().prefetch(AUTOTUNE) model = keras.Sequential([ layers.Resizing(128,128, interpolation= 'nearest'), layers.Conv2D(filters= 32, activation= 'relu', padding= 'same', strides= 1, kernel_size= 3), layers.MaxPooling2D(), layers.Conv2D(filters= 64, activation= 'relu', padding= 'same', strides= 1, kernel_size= 3), layers.Conv2D(filters= 64, activation= 'relu', padding= 'same', strides= 1, kernel_size= 3), layers.MaxPooling2D(), layers.Conv2D(filters= 128, activation= 'relu', padding= 'same', strides= 1, kernel_size= 3), layers.Conv2D(filters= 128, activation= 'relu', padding= 'same', strides= 1, kernel_size= 3), layers.MaxPooling2D(), layers.Flatten(), layers.Dense(32, activation= 'relu', kernel_regularizer= keras.regularizers.L2(0.001)), layers.Dropout(0.4), layers.Dense(10, activation= 'softmax') ]) model.compile(optimizer= 'adam', loss= keras.losses.CategoricalCrossentropy(), metrics= ['categorical_crossentropy','accuracy']) early_stop = callbacks.EarlyStopping(min_delta= 0.001, patience= 15, restore_best_weights= True, monitor= 'val_categorical_crossentropy') history= model.fit(train_data, validation_data= test_data, epochs= 300, callbacks= [early_stop])
电脑配置
Windows 10
Intel Core i3-8100
AMD Radeon RX580(使用Tensorflow DirectML插件)
16 GB 内存
请问有没有无需关闭eager execution的解决办法?
内容的提问来源于stack exchange,提问作者Delinester
相关产品推荐
相关产品推荐

