PyTorch训练模型与Intel OpenVINO预训练模型结合异常求助

我在Google Colab中用PyTorch训练了头盔识别模型，准确率接近96%。按照Intel OpenVINO官方文档将模型转为IR格式后，和预训练的pedestrian-detection-adas-0002行人检测模型结合，搭建目标检测+识别架构，但组合后的系统无法正常工作。

运行代码 #

def PeopleBox(PeopleNet,frame):
    frameHeight=frame.shape[0]
    frameWidth=frame.shape[1]
    blob=cv2.dnn.blobFromImage(frame, 1.0, (672,384), swapRB=False, crop=True)
    PeopleNet.setInput(blob)
    detection=PeopleNet.forward()
    bboxs=[]
    for i in range(detection.shape[2]):
        confidence=detection[0,0,i,2]
        if confidence>0.7:
            x1=int(detection[0,0,i,3]*frameWidth)
            y1=int(detection[0,0,i,4]*frameHeight)
            x2=int(detection[0,0,i,5]*frameWidth)
            y2=int(detection[0,0,i,6]*frameHeight)
            bboxs.append([x1,y1,x2,y2])
            cv2.rectangle(frame, (x1,y1),(x2,y2),(0,255,0), 8)
    return frame, bboxs

PeopleBin = (r"C:\Users\directory\T1\pedestrian-detection-adas-0002.bin")
PeopleXml = (r"C:\Users\directory\T1\pedestrian-detection-adas-0002.xml")
HelmetBin = (r"C:\Users\dc\StructVGGV1_output\VGG16_V1_40epohc_LR0_00008_batch4_A96_V77.bin")
HelmetXml = (r"C:\Users\dc\StructVGGV1_output\VGG16_V1_40epohc_LR0_00008_batch4_A96_V77.xml")
PeopleNet=cv2.dnn.readNet(PeopleXml, PeopleBin)
HelmetNet=cv2.dnn.readNet(HelmetXml,HelmetBin)
List = ['NoPersonHoldingHelmet', 'PersonHoldingHelmet']
video=cv2.VideoCapture(0)

while True:
    ret,frame=video.read()
    framee,bboxs=PeopleBox(PeopleNet,frame)
    for bbox in bboxs:
        # 错误：直接用整帧生成blob，未裁剪行人区域
        blob=cv2.dnn.blobFromImage(framee, 1.0, (224,224), swapRB=False, crop = True) 
        HelmetNet.setInput(blob)
        HelmetPred=HelmetNet.forward()
        Helmet=List[HelmetPred[0].argmax()]
        label="{}".format(Helmet)
        if label=="NoPersonHoldingHelmet":
            cv2.rectangle(framee, (bbox[0], bbox[1]), (bbox[2],bbox[3]), (255,0,255), 8)
        if label == "PersonHoldingHelmet":
            cv2.rectangle(framee, (bbox[0], bbox[1]), (bbox[2],bbox[3]), (255,255,0), 8)
            
        cv2.putText(framee, label, (bbox[0]-70, bbox[1]-10), cv2.FONT_HERSHEY_SIMPLEX, 0.8, (255,255,255), 2,cv2.LINE_AA)

    # 错误：重复调用imshow导致窗口异常
    cv2.imshow("Helmet_Vs_NoHelmet",framee)
    cv2.imshow("Helmet_Vs_NoHelmet",framee)
    k=cv2.waitKey(1)
    if k==ord('q'):
        break
video.release()
cv2.destroyAllWindows()

核心问题排查与修复 #

• 头盔模型输入未裁剪行人区域
  代码中直接用整帧图像喂给头盔识别模型，和训练时输入（仅包含行人/带头盔的行人）完全不符，这是最致命的错误。修正方式：裁剪行人检测出的bbox区域作为头盔模型输入：

  # 替换原blob生成代码
  x1, y1, x2, y2 = bbox
  # 防止坐标越界
  x1 = max(0, x1)
  y1 = max(0, y1)
  x2 = min(framee.shape[1], x2)
  y2 = min(framee.shape[0], y2)
  # 裁剪行人ROI区域
  roi = framee[y1:y2, x1:x2]
  blob = cv2.dnn.blobFromImage(roi, 1.0, (224,224), swapRB=False, crop=True)
  

• 预处理参数与训练时不匹配
  检查PyTorch训练时的图像预处理逻辑：

  • 若训练时做了归一化（如除以255、减去均值），需在blobFromImage中添加对应scalefactor和mean参数
  • 若训练用RGB通道输入，OpenCV读取的是BGR，需将swapRB设为True

• 模型转换验证
  用OpenVINO的Model Optimizer重新转换模型，确保指定了正确的输入形状（如--input_shape [1,3,224,224]），且转换日志无报错。

• 行人检测阈值调整
  当前行人检测置信度阈值设为0.7，可临时降低到0.5测试，确认行人检测是否能正常输出bbox。

• 重复窗口显示修复
  删除其中一行cv2.imshow("Helmet_Vs_NoHelmet",framee)，避免窗口刷新异常。

内容的提问来源于stack exchange，提问作者newbieLife