迁移学习扩展ResNet50类别时丢失原有类别，如何实现单模型兼容新旧类别

Hey there, let's break down exactly why your original code lost all the pre-trained ImageNet classes, and walk through how to build a single model that handles both the original 1000 classes and your 4 new ones.

What Went Wrong in Your Original Code #

Your initial setup completely replaced the ResNet50's original output layer (the predictions dense layer that outputs 1000 classes) with your own 4-class dense layer. That's why all the original categories disappeared—you essentially threw away that part of the model.

When you tried the multi-output trick (output=[out, model.output]), you didn't adjust your loss function to match the two separate outputs. Keras had no idea how to map a single categorical crossentropy loss to outputs of shapes (None,4) and (None,1000), hence the shape mismatch error.

Step-by-Step Solution: Multi-Output Model with Combined Loss #

We'll build a model that keeps the original ImageNet output branch AND adds your new 4-class branch, then set up training to handle both tasks.

1. Define the Multi-Output Model Structure #

First, load the pre-trained ResNet50 with its original output layer intact, then add your new branch off the avg_pool layer (same as you did before, but we won't discard the original output):

from tensorflow.keras.applications import ResNet50
from tensorflow.keras.layers import Input, Flatten, Dense
from tensorflow.keras.models import Model

# Define your input shape (adjust if your images are a different size)
image_input = Input(shape=(224, 224, 3))

# Load ResNet50 with pre-trained ImageNet weights, keeping the original 1000-class output
base_model = ResNet50(input_tensor=image_input, include_top=True, weights='imagenet')

# Add your new 4-class branch, using the avg_pool layer as the starting point
last_layer = base_model.get_layer('avg_pool').output
x = Flatten(name='flatten')(last_layer)
new_classes_output = Dense(4, activation='softmax', name='new_classes_output')(x)

# Create a model that outputs BOTH the original 1000 classes and your new 4 classes
custom_resnet_model = Model(inputs=image_input, outputs=[base_model.output, new_classes_output])
custom_resnet_model.summary()

2. Configure Trainable Layers #

Decide which layers to freeze/unfreeze. If you want to preserve the original ImageNet performance, you can freeze most of the base model and only train the original output layer (optional) and your new branch:

# Freeze all base model layers except the final predictions layer (adjust if you want more fine-tuning)
for layer in base_model.layers[:-1]:
    layer.trainable = False

# Make sure the original output layer and your new branch are trainable
base_model.layers[-1].trainable = True
custom_resnet_model.get_layer('new_classes_output').trainable = True

If you don't care about fine-tuning the original classes, you can freeze the entire base model (including predictions) and only train your new 4-class branch.

3. Compile with Multi-Output Loss #

Since we have two outputs, we need to specify a loss function for each, plus optional weights to balance the two tasks (e.g., prioritize your new classes more):

custom_resnet_model.compile(
    # Map each output layer to its loss function
    loss={'predictions': 'categorical_crossentropy', 'new_classes_output': 'categorical_crossentropy'},
    # Assign weights to balance loss contributions (tweak these based on your priorities)
    loss_weights={'predictions': 0.3, 'new_classes_output': 0.7},
    optimizer='adam',
    # Track accuracy for both tasks
    metrics={'predictions': ['accuracy'], 'new_classes_output': ['accuracy']}
)

Alternatively, you can use a list format (order must match your model's output order):

custom_resnet_model.compile(
    loss=['categorical_crossentropy', 'categorical_crossentropy'],
    loss_weights=[0.3, 0.7],
    optimizer='adam',
    metrics=['accuracy']
)

4. Prepare Training Data & Train #

Your training data needs to provide labels for both tasks:

• Original ImageNet labels: 1000-dimensional one-hot vectors (use the official ImageNet labels for existing classes, or pseudo-labels generated by the pre-trained model if you don't have ground truth)
• New class labels: 4-dimensional one-hot vectors for your 4 new categories

Train using a dictionary or list of labels (matching your compile setup):

# Example using dictionary labels (matches the first compile style)
history = custom_resnet_model.fit(
    train_images,
    {
        'predictions': train_labels_imagenet,
        'new_classes_output': train_labels_new
    },
    epochs=10,
    batch_size=32,
    validation_split=0.2
)

Key Notes #

• If you don't have ground truth labels for the original ImageNet classes, you can generate pseudo-labels by running the pre-trained ResNet50 on your dataset and using those predictions as labels for the original branch. This won't be perfect, but it will help preserve the original class performance.
• Adjust the loss_weights to prioritize either the original classes or your new ones based on your use case.
• When making predictions, the model will return two arrays: the first is the 1000-class probabilities, the second is your 4-class probabilities.

内容的提问来源于stack exchange，提问作者Coder