如何基于已训练CNN模型实现视频目标检测并绘制边界框

你当前训练的是纯全连接的图像分类网络，仅支持对整张输入图像输出分类结果，不具备目标定位能力，无法直接输出边界框坐标。要实现视频目标检测，可参考以下两种实现方案：

采用滑窗法+图像金字塔的逻辑实现定位，具体实现步骤：

• 对视频读取的每一帧生成图像金字塔：按不同比例缩放帧画面，适配不同尺寸的待检测目标
• 用固定尺寸的滑动窗口在每一层缩放后的帧上遍历滑动，将窗口内容resize到训练时的[width, height]尺寸，做除以255的归一化处理后送入你训练好的模型做分类预测
• 设置置信度阈值，保留预测得分高于阈值的窗口，再通过非极大值抑制（NMS）过滤重叠度高的重复框，得到最终的目标边界框坐标
• 调用OpenCV的cv2.rectangle()方法在原帧上绘制边界框，用cv2.putText()标注目标类别和置信度

对应示例代码：

import cv2
import numpy as np
import tensorflow as tf

# 替换为你训练时的实际输入尺寸
width = 28
height = 28
# 替换为你实际的类别标签对应表
class_names = ["类别1", "类别2", ..., "类别10"]

# 图像金字塔生成函数
def pyramid(image, scale=1.5, min_size=(width, height)):
    yield image
    while True:
        w = int(image.shape[1] / scale)
        h = int(image.shape[0] / scale)
        image = cv2.resize(image, (w, h))
        if image.shape[0] < min_size[1] or image.shape[1] < min_size[0]:
            break
        yield image

# 滑窗函数
def sliding_window(image, step_size, window_size):
    for y in range(0, image.shape[0] - window_size[1], step_size):
        for x in range(0, image.shape[1] - window_size[0], step_size):
            yield (x, y, image[y:y + window_size[1], x:x + window_size[0]])

# NMS非极大值抑制函数
def non_max_suppression(boxes, scores, threshold=0.3):
    if len(boxes) == 0:
        return []
    boxes = np.array(boxes)
    scores = np.array(scores)
    x1 = boxes[:, 0]
    y1 = boxes[:, 1]
    x2 = boxes[:, 2]
    y2 = boxes[:, 3]
    areas = (x2 - x1 + 1) * (y2 - y1 + 1)
    order = scores.argsort()[::-1]
    keep = []
    while order.size > 0:
        i = order[0]
        keep.append(i)
        xx1 = np.maximum(x1[i], x1[order[1:]])
        yy1 = np.maximum(y1[i], y1[order[1:]])
        xx2 = np.minimum(x2[i], x2[order[1:]])
        yy2 = np.minimum(y2[i], y2[order[1:]])
        w = np.maximum(0, xx2 - xx1 + 1)
        h = np.maximum(0, yy2 - yy1 + 1)
        overlap = (w * h) / areas[order[1:]]
        inds = np.where(overlap <= threshold)[0]
        order = order[inds + 1]
    return keep

# 加载训练好的模型，已保存模型可以用下行代码加载，未保存直接用训练好的model变量即可
# model = tf.keras.models.load_model("你的模型路径.h5")
model = 你的训练好的模型变量

# 读取视频
cap = cv2.VideoCapture('cars.mp4')
# 滑动窗口步长，可根据实际效果调整
step_size = 10
# 置信度阈值，可根据实际效果调整
conf_threshold = 0.7

while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break
    boxes = []
    scores = []
    class_ids = []
    # 遍历图像金字塔每一层
    for resized in pyramid(frame):
        scale = frame.shape[1] / resized.shape[1]
        # 遍历滑窗
        for (x, y, window) in sliding_window(resized, step_size=step_size, window_size=(width, height)):
            if window.shape[0] != height or window.shape[1] != width:
                continue
            # 预处理窗口图像
            window = window / 255.0
            window = np.expand_dims(window, axis=0)
            # 预测
            pred = model.predict(window, verbose=0)[0]
            class_id = np.argmax(pred)
            score = pred[class_id]
            if score > conf_threshold:
                # 坐标映射回原帧尺寸
                x1 = int(x * scale)
                y1 = int(y * scale)
                x2 = int((x + width) * scale)
                y2 = int((y + height) * scale)
                boxes.append([x1, y1, x2, y2])
                scores.append(score)
                class_ids.append(class_id)
    # 执行NMS过滤重复框
    keep = non_max_suppression(boxes, scores)
    # 绘制边界框和类别
    for i in keep:
        x1, y1, x2, y2 = boxes[i]
        cv2.rectangle(frame, (x1, y1), (x2, y2), (0, 255, 0), 2)
        text = f"{class_names[class_ids[i]]}: {scores[i]:.2f}"
        cv2.putText(frame, text, (x1, y1-10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 2)
    # 显示结果
    cv2.imshow("Detection", frame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

注意：你当前的分类模型没有卷积层，特征提取能力较弱，滑窗法的检测精度会比较差，且实时性很低，仅可用于简单测试场景。

要实现精度和速度都可用的视频目标检测，建议直接使用专门的目标检测架构，比如YOLO系列、Faster R-CNN等，这类模型本身就集成了分类和定位能力，可直接输出每个目标的类别、置信度和边界框坐标。你可以用自己的数据集对预训练的检测模型做微调训练，即可适配你的目标类别需求。

内容的提问来源于stack exchange，提问作者Tawfiq Chowdhury