基于TensorFlow自定义输入尺寸的InceptionV3迁移学习遇错求助

Hey there! Let's tackle this problem step by step—this is a common gotcha when working with pre-trained models like InceptionV3, so you're not alone. The core issue here is that the full pre-trained InceptionV3 (with the top classification layer) has fixed input dimensions tied to its final fully connected layers. Here's how to fix it without using resize:

1. Load InceptionV3 without the pre-trained top layer #

When you download the default InceptionV3 from TensorFlow's Keras Applications, it includes a pre-trained classification head (fully connected layers) that expects 299x299 input. To use custom input sizes, you need to exclude this top layer and define your own:

import tensorflow as tf

# Load the base model without the top classification layer
base_model = tf.keras.applications.InceptionV3(
    weights='imagenet',  # Retain pre-trained ImageNet weights
    include_top=False,   # Exclude the fixed-dimension top layer
    input_shape=(512, 512, 3)  # Define your custom input size here
)

This tells TensorFlow to initialize the base convolutional layers to accept your 512x512x3 input, instead of the hardcoded 299x299.

2. Freeze the base model (for initial transfer learning) #

To start with transfer learning, freeze the base convolutional layers so you don't overwrite the pre-trained features while training your custom top layer:

base_model.trainable = False

If you want to fine-tune later (after training the top layer), you can unfreeze specific layers of the base model (more on that below).

3. Build your custom classification/regression head #

Since we removed the original top layer, we need to add our own that works with any input size. Using GlobalAveragePooling2D is perfect here—it collapses the spatial dimensions of the convolutional output into a fixed-size vector, regardless of the input image size:

# Get the output of the base model
x = base_model.output

# Add global average pooling to create a fixed-size feature vector
x = tf.keras.layers.GlobalAveragePooling2D()(x)

# Add a dense hidden layer (adjust units based on your task complexity)
x = tf.keras.layers.Dense(1024, activation='relu')(x)

# Add your final output layer (adjust num_classes to match your dataset)
num_classes = 10  # Replace with your actual number of classes
predictions = tf.keras.layers.Dense(num_classes, activation='softmax')(x)

# Combine the base model and custom head into a single trainable model
model = tf.keras.models.Model(inputs=base_model.input, outputs=predictions)

4. Compile and train the model #

Now you can compile and train this model directly on your 512x512 dataset, no resizing needed:

model.compile(
    optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),
    loss='categorical_crossentropy',  # Adjust loss function based on your task (e.g., binary crossentropy for 2 classes)
    metrics=['accuracy']
)

# Train on your custom dataset
model.fit(
    train_dataset,  # Your 512x512 image dataset
    epochs=10,
    validation_data=val_dataset
)

5. Optional: Fine-tune the base model (for better performance) #

Once your custom top layer is trained, you can unfreeze parts of the base model to fine-tune the pre-trained features to your dataset. Make sure to use a very small learning rate to avoid destroying the pre-trained weights:

# Unfreeze the top N layers of the base model
base_model.trainable = True
fine_tune_at = 249  # Example: unfreeze layers from index 249 onwards (adjust based on your needs)

for layer in base_model.layers[:fine_tune_at]:
    layer.trainable = False

# Recompile with a smaller learning rate for fine-tuning
model.compile(
    optimizer=tf.keras.optimizers.Adam(learning_rate=1e-5),
    loss='categorical_crossentropy',
    metrics=['accuracy']
)

# Continue training with fine-tuning
model.fit(
    train_dataset,
    epochs=20,  # Add more epochs for fine-tuning
    initial_epoch=model.epoch,
    validation_data=val_dataset
)

Why your previous attempt failed #

If you tried loading the full InceptionV3 (with include_top=True) and changing the input size, the pre-trained fully connected layers would expect a fixed-size feature map from the base model. For 299x299 input, the base model outputs an 8x8x2048 feature map—but for 512x512 input, it outputs a 16x16x2048 feature map, which doesn't match the dimensions the original top layer was trained for. Excluding the top layer fixes this mismatch entirely.

内容的提问来源于stack exchange，提问作者Assassin