如何基于已训练的Keras分类卷积网络实现多目标检测？

Great job getting your classification model to 93% accuracy on the 10-class dataset! Let’s break down how to adapt this model for multi-object detection, and whether retraining is needed.

First, a key note: your current model is an image classification model—it only outputs a single class label for the entire image, with no ability to locate multiple objects. To do detection, we need to modify it or pair it with additional logic. Here are your two main options:

Option 1: Sliding Window + Pre-trained Classification Model (No Retraining Needed, Less Efficient) #

This approach leverages your existing model by running it on small, sliding sub-windows of the input image to find and classify objects. It works without retraining, but is slower and less precise than dedicated detection models.

How it works: #

1. Generate candidate windows: Create a set of windows with varying sizes (e.g., 64x64, 128x128, 256x256) and slide them across the image at fixed step sizes (e.g., 32 pixels).
2. Classify each window: Resize/normalize each sub-window to match your model’s input requirements, then pass it through your pre-trained model to get a class label and confidence score.
3. Filter and clean results: Remove low-confidence predictions (e.g., below 0.7) and use Non-Maximum Suppression (NMS) to eliminate overlapping duplicate boxes.

Simplified Code Example: #

import numpy as np
import tensorflow as tf

def sliding_window(image, step_size, window_size):
    # Slide a window across the input image
    for y in range(0, image.shape[0] - window_size[1], step_size):
        for x in range(0, image.shape[1] - window_size[0], step_size):
            yield (x, y, image[y:y + window_size[1], x:x + window_size[0]])

# Load and preprocess your input image (match training preprocessing)
input_img = ...  # Load your image (e.g., via tf.keras.utils.load_img)
preprocessed_img = tf.keras.applications.imagenet_utils.preprocess_input(input_img)
preprocessed_img = np.expand_dims(preprocessed_img, axis=0)  # Add batch dimension

# Define window parameters
window_sizes = [(64, 64), (128, 128), (256, 256)]
step_size = 32
confidence_threshold = 0.7

detections = []

for win_size in window_sizes:
    for (x, y, window) in sliding_window(preprocessed_img[0], step_size, win_size):
        # Skip windows that don't match the target size
        if window.shape[0] != win_size[1] or window.shape[1] != win_size[0]:
            continue
        # Prepare window for model input
        window_input = np.expand_dims(window, axis=0)
        # Get predictions
        preds = model.predict(window_input)
        class_idx = np.argmax(preds[0])
        confidence = preds[0][class_idx]
        # Keep high-confidence results
        if confidence > confidence_threshold:
            detections.append((x, y, x+win_size[0], y+win_size[1], class_idx, confidence))

# Apply Non-Maximum Suppression to remove overlapping boxes
def apply_nms(detections, iou_threshold=0.5):
    boxes = np.array([[x1, y1, x2, y2] for (x1,y1,x2,y2,_,_) in detections])
    scores = np.array([conf for (_,_,_,_,_,conf) in detections])
    selected_indices = tf.image.non_max_suppression(
        boxes, scores, max_output_size=100, iou_threshold=iou_threshold
    )
    return [detections[i] for i in selected_indices.numpy()]

final_detections = apply_nms(detections)
# final_detections contains (x1, y1, x2, y2, class_id, confidence) for each detected object

Option 2: Transfer Learning to Build a Dedicated Detection Model (Requires Retraining, Better Performance) #

Your pre-trained model’s convolutional layers already learn powerful image features—we can repurpose these as a feature extractor for a detection model. This requires retraining, but will give you far better speed and accuracy than the sliding window method.

How it works: #

1. Freeze the convolutional base: Lock the weights of your existing model’s convolutional layers (they already know how to extract useful image features).
2. Replace classification layers with detection heads: Swap out the final dense/classification layers for layers that predict bounding box coordinates and class labels for multiple objects (e.g., using SSD, YOLO, or Faster R-CNN-style outputs).
3. Train with labeled detection data: You’ll need a dataset where each image has annotated bounding boxes and class labels for all objects. Train only the new detection head layers (or fine-tune the convolutional base if you have enough data).

Simplified Code Example (SSD-style Detection Head): #

# Extract the convolutional base from your pre-trained model (remove final dense layers)
conv_base = tf.keras.Model(inputs=model.input, outputs=model.get_layer("conv2d_2").output)
conv_base.trainable = False  # Freeze the convolutional layers

# Add detection head layers to predict bounding boxes and classes
x = conv_base.output
# Predict bounding boxes (4 coordinates per object: x, y, width, height) for 10 classes + background
bbox_predictions = tf.keras.layers.Conv2D(4 * (10 + 1), (3, 3), padding="same", name="bbox_pred")(x)
# Predict class probabilities (10 classes + background)
class_predictions = tf.keras.layers.Conv2D(10 + 1, (3, 3), padding="same", activation="softmax", name="class_pred")(x)

# Build the detection model
detection_model = tf.keras.Model(inputs=conv_base.input, outputs=[bbox_predictions, class_predictions])

# Define a custom loss function for detection
def detection_loss(y_true, y_pred):
    bbox_true, class_true = y_true
    bbox_pred, class_pred = y_pred
    
    # Only calculate loss for non-background objects
    mask = tf.cast(tf.not_equal(class_true, 10), tf.float32)  # 10 = background class
    
    # Bounding box regression loss (MSE)
    bbox_loss = tf.reduce_mean(mask * tf.square(bbox_pred - bbox_true))
    # Classification loss (cross-entropy)
    class_loss = tf.reduce_mean(mask * tf.keras.losses.sparse_categorical_crossentropy(class_true, class_pred))
    
    return bbox_loss + class_loss

# Compile and train the model with your detection dataset
detection_model.compile(optimizer=tf.keras.optimizers.Adam(1e-4), loss=detection_loss)
# detection_model.fit(detection_dataset, epochs=20, validation_data=val_dataset)

Key Takeaways: #

• No retraining: Use the sliding window method for quick testing, but expect slower speeds and less precise detection (especially for small objects).
• Retraining required: For production-quality multi-object detection, repurpose your convolutional base with a detection head. This uses your existing feature knowledge and gives far better results.

内容的提问来源于stack exchange，提问作者Amanda