TensorFlow多GPU模型初始化报错及GPU指定问题咨询

Hey there, let's break down your issues and fix them step by step—those errors you're hitting are super common when using the old multi_gpu_model API with TensorFlow 2.x (especially with Eager Execution turned on).

一、Why those errors happen #

First off, the multi_gpu_model API is actually deprecated in TensorFlow 2.x. That's why you're seeing those weird type errors:

• TypeError: int() argument must be a string or a number, not 'TensorShape': The old implementation wasn't built to handle Eager Execution's TensorShape handling, so it chokes when splitting model parameters.
• AttributeError: 'DeferredTensor' object has no attribute 'device': Even switching to Keras' native multi_gpu_model won't help because TensorFlow 2.x changed how internal tensors work, and the old API can't recognize the new DeferredTensor type.
• The ValueError when not specifying GPU count? That's the old API trying to auto-detect all visible GPUs, but it probably misjudged the number available due to your PCI slot setup.

二、The official fix: Use tf.distribute.MirroredStrategy #

TensorFlow 2.x recommends using Distributed Strategies for multi-GPU training—it's fully compatible with Eager Execution, way more flexible, and the future-proof approach. Here's how to adapt your code:

1. Basic multi-GPU setup (with custom GPU selection) #

import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Flatten, Dense

# First, pick which GPUs to use
# Get list of physical GPUs (matches your PCI slots 1 and 2, usually gpus[0] and gpus[1])
gpus = tf.config.experimental.list_physical_devices('GPU')
if gpus:
    # Restrict TensorFlow to only see the two GPUs you want
    tf.config.experimental.set_visible_devices([gpus[0], gpus[1]], 'GPU')
    # Initialize mirrored strategy with your target GPUs
    strategy = tf.distribute.MirroredStrategy(devices=["/gpu:0", "/gpu:1"])

# Create and compile your model INSIDE the strategy scope
with strategy.scope():
    model = Sequential()
    model.add(Flatten(input_shape=(128, 128, 3)))
    model.add(Dense(100, activation="sigmoid"))
    model.add(Dense(100, activation="sigmoid"))
    
    # Compile must happen here too
    model.compile(optimizer='adam', loss='mse')

# Train just like you would with a single GPU
model.fit(x_train, y_train, epochs=10, batch_size=64)

2. Custom GPU selection explained #

• tf.config.experimental.list_physical_devices('GPU') gives you a list of all physical GPUs connected to your system. Your PCI slot 1 will typically be gpus[0], slot 2 will be gpus[1] (you can verify this with nvidia-smi if you're on NVIDIA).
• tf.config.experimental.set_visible_devices tells TensorFlow to only use the GPUs you specify, so it won't try to access others.
• MirroredStrategy(devices=["/gpu:0", "/gpu:1"]) ensures the model is replicated across exactly those two GPUs for parallel training.

三、If you REALLY need to use multi_gpu_model (not recommended) #

If you have to stick with the old API for some reason, you'll need to disable Eager Execution (since it's incompatible with multi_gpu_model in TF 2.x):

import tensorflow as tf
# Turn off Eager Execution (loses TF 2.x core features, so not ideal)
tf.compat.v1.disable_eager_execution()

from tensorflow.python.keras.utils import multi_gpu_model
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Flatten, Dense

model = Sequential()
model.add(Flatten(input_shape=(128, 128, 3)))
model.add(Dense(100, activation="sigmoid"))
model.add(Dense(100, activation="sigmoid"))

# Specify 2 GPUs
model = multi_gpu_model(model, gpus=2, cpu_merge=False)
model.compile(optimizer='adam', loss='mse')

But seriously, avoid this if you can—Eager Execution makes debugging and iterative development way easier, and Distributed Strategies are the supported path forward.

内容的提问来源于stack exchange，提问作者thebeancounter