Blackfly S相机1440x1080分辨率帧率瓶颈排查求助

云上老周

2026-6-19

问题背景与排查请求

设备与任务概述

使用两台Blackfly S BFS-U3-16S2M相机进行蚊子运动检测，相机通过GPIO同步。检测到运动后，触发相机持续录制、读取循环缓冲区并保存视频。采用EasyPySpin库控制相机，结合OpenCV实现运动检测——未使用官方Spinnaker库，因无法解决其图像格式与OpenCV的兼容问题。

核心问题

官方规格显示该相机在1440x1080分辨率Mono8格式下可达226fps，但实际该分辨率下连60fps都无法实现：录制600帧需12-13秒（60fps下理论仅需10秒）。

系统负载

CPU使用率峰值约550%
内存使用率峰值20%

已尝试方案

使用线程：录制600帧时间缩短至11-12秒，但线程实现可能存在问题
降低分辨率：帧率明显提升，但需求是在1440x1080分辨率下达到约200fps

软硬件配置

主板：B650 AORUS ELITE AX V2
GPU：RTX 4070
处理器：Ryzen 7 7700x
系统：Ubuntu 22.04.4
Python版本：3.10.12
相关库：Spinnaker 4.0.0.116、EasyPySpin 2.0.1、OpenCV-Python 4.10.0.84等

当前运动检测脚本

import cv2
import EasyPySpin
import time
from time import sleep
import numpy as np
from datetime import datetime
from collections import deque

# OS Version: Ubuntu 22.04.4
# Python: 3.10.12
# spinnaker: 4.0.0.116
# spinnaker-python: 4.0.0.116
# EasyPySpin: 2.0.1
# opencv-python: 4.10.0.84
# numpy: 1.26.4


# BEFORE SYNCHRONIZATION:
# 500x500 works with 200fps
# 1440x1080 works with <=75fps

# AFTER SYNCHRONIZATION:
# 1440X1080 with <= 40fps

serial_number_0 = "24122966"  # primary camera serial number
serial_number_1 = "24122965"  # secondary camera serial number
cap = EasyPySpin.SynchronizedVideoCapture(serial_number_0, serial_number_1)
cap.set(cv2.CAP_PROP_FPS, 100)
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 1280)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 720)
fourcc = cv2.VideoWriter_fourcc(*'XVID')
out_0 = None
out_1 = None
buffer_size = 1000
additional_frame_size = 1000
additional_frames_0 = deque(maxlen=additional_frame_size)
additional_frames_1 = deque(maxlen=additional_frame_size)
ring_buffer_0 = deque(maxlen=buffer_size)
ring_buffer_1 = deque(maxlen=buffer_size)
cap.set(cv2.CAP_PROP_EXPOSURE, 100)
print(cap.get(cv2.CAP_PROP_EXPOSURE))

def detect_motion(frame, back_sub, kernel, min_contour_area, i):
    fg_mask = back_sub.apply(frame)
    fg_mask = cv2.morphologyEx(fg_mask, cv2.MORPH_CLOSE, kernel)
    fg_mask = cv2.medianBlur(fg_mask, 5)
    _, fg_mask = cv2.threshold(fg_mask, 127, 255, cv2.THRESH_BINARY)

    contours, _ = cv2.findContours(fg_mask, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
    areas = [cv2.contourArea(c) for c in contours]

    if len(areas) == 0:
        cv2.imshow(f"frame-{i}", frame)
        if cv2.waitKey(1) == ord('q'):
            return None
    else:
        max_index = np.argmax(areas)
        if areas[max_index] > min_contour_area:
            return contours[max_index]
    return None

def motion_detection():
    sleep(5)
    print("Starting motion detection. Press Ctrl+C to stop.")
    global out_0
    global out_1
    prev_x = None
    recording = False
    frame_counter = 0

    # INCREASE varThreshold = LESS SENSITIVE MOTION DETECTION
    back_sub = cv2.createBackgroundSubtractorMOG2(history=700, varThreshold=50, detectShadows=True)
    # INCREASE KERNEL SIZE FOR MORE AGGRESSIVE NOISE REDUCTION
    kernel = np.ones((30, 30), np.uint8)
    # DETERMINES THE CONTOUR SIZE TO BE CONSIDERED AS VALID MOTION
    # ONLY CONTOURS WITH AN AREA OF 1000 PIXELS OR MORE WILL BE CONSIDERED AS VALID MOTION.
    min_contour_area = 100

    while True:
        read_values = cap.read()
        for i, (ret, frame) in enumerate(read_values):
            if not ret:
                print("Error: Failed to capture image")
                break
            if not recording:
                if i == 0:
                    ring_buffer_0.append(frame)
                elif i == 1:
                    ring_buffer_1.append(frame)

            frame_copy = np.copy(frame)
            if not recording:
                contour = detect_motion(frame_copy, back_sub, kernel, min_contour_area, i)

            if contour is not None:
                x, y, w, h = cv2.boundingRect(contour)
                x2 = x + int(w / 2)
                y2 = y + int(h / 2)

                cv2.rectangle(frame_copy, (x, y), (x + w, y + h), (0, 255, 0), 3)
                cv2.circle(frame_copy, (x2, y2), 4, (0, 255, 0), -1)
                text = f"x: {x2}, y: {y2}"
                cv2.putText(frame_copy, text, (x2 - 10, y2 - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 2)

                cv2.imshow(f"frame-{i}", frame_copy)
                if cv2.waitKey(1) == ord('q'):
                    break

                if prev_x is not None and x2 < prev_x and not recording:
                    print("Motion Detected!")
                    print("Start: ", datetime.now().strftime("%Y%-m-%d_%H:%M:%S.%f")[:-3])
                    start_time = time.time()
                    recording = True
                    frame_counter = 0
                    frame_height, frame_width = frame.shape[:2]
                    timestamp = datetime.now().strftime("%Y%m%d_%H%M%S_")
                    print(frame_width, frame_height)
                    out_0 = cv2.VideoWriter(f'motion_detection_{timestamp}_0.avi', fourcc, 100.0, (frame_width, frame_height), isColor=False)
                    out_1 = cv2.VideoWriter(f'motion_detection_{timestamp}_1.avi', fourcc, 100.0, (frame_width, frame_height), isColor=False)
                prev_x = x2

            if recording:
                if i == 0:
                    additional_frames_0.append(frame)
                elif i == 1:
                    additional_frames_1.append(frame)
                frame_counter += .5 
                if frame_counter >= additional_frame_size:
                    recording = False
                    print("End: ", datetime.now().strftime("%Y%-m-%d_%H:%M:%S.%f")[:-3])
                    print("Finished recording. Retrieving buffer and saving video...")
                    print("Elapsed time:", time.time() - start_time)
                    for frame in ring_buffer_0:
                        out_0.write(frame)
                    for frame in additional_frames_0:
                        out_0.write(frame)
                    for frame in ring_buffer_1:
                        out_1.write(frame)
                    for frame in additional_frames_1:
                        out_1.write(frame)
                    out_0.release()
                    out_1.release()
                    print("Video Saved!")
                    additional_frames_0.clear()
                    additional_frames_1.clear()
                    ring_buffer_0.clear()
                    ring_buffer_1.clear()

if __name__ == '__main__':
    try:    
        motion_detection()
    except KeyboardInterrupt:
        print("Stopping motion detection.")
    finally:
        cap.release()
        if out_1 is not None and out_0 is not None:
            out_1.release()
            out_0.release()
        cv2.destroyAllWindows()
        print("Closing camera and resetting...")

瓶颈排查与优化建议

1. 相机配置与数据传输优化

强制Mono8格式输出：手动指定相机输出格式为Mono8，避免格式转换开销：
```
cap.set(cv2.CAP_PROP_FORMAT, cv2.CAP_MODE_MONO8)
```
检查USB带宽：两台相机1440x1080@200fps Mono8的总带宽约为620MB/s，接近USB3.0理论上限（625MB/s），可将目标帧率设为200fps，同时确保使用合格的USB3.0线缆，且主板USB口为USB3.2 Gen1及以上规格。
固定相机参数：关闭自动曝光、自动增益等自动功能，手动固定参数，减少相机内部计算开销。

2. 运动检测算法优化

简化形态学操作：将MORPH_CLOSE替换为MORPH_OPEN先去噪再闭运算，同时缩小kernel尺寸（如改为(15,15)）；将medianBlur(5)替换为GaussianBlur(5,5)，提升处理速度。

优化轮廓检测：用cv2.RETR_EXTERNAL代替cv2.RETR_TREE，仅检测最外层轮廓；直接遍历轮廓跟踪最大面积目标，避免numpy数组遍历开销：

max_area = 0
max_contour = None
for cnt in contours:
    area = cv2.contourArea(cnt)
    if area > max_area:
        max_area = area
        max_contour = cnt
if max_area > min_contour_area:
    return max_contour

GPU加速处理：利用RTX4070的CUDA能力，将所有图像处理操作迁移到GPU：使用cv2.cuda_GpuMat上传帧，调用CUDA版本的背景减除、形态学操作等函数，大幅降低CPU负载。

3. 缓冲区与视频写入优化

多线程视频写入：开启独立线程负责两台相机的视频写入，主线程仅将帧放入线程安全队列，避免IO阻塞影响帧率。
更换高效编码格式：将XVID替换为MJPG或H.264编码，提升编码速度：
```
fourcc = cv2.VideoWriter_fourcc(*'MJPG')  # 或'H264'（需系统支持）
```

numpy环形缓冲区：用numpy数组替代deque作为环形缓冲区，减少内存分配开销：

ring_buffer_0 = np.zeros((buffer_size, 1080, 1440), dtype=np.uint8)
buffer_idx_0 = 0
# 写入缓冲区
ring_buffer_0[buffer_idx_0] = frame
buffer_idx_0 = (buffer_idx_0 + 1) % buffer_size

4. 线程架构优化

拆分任务为三个独立线程：

相机读取线程：专门负责从同步相机读取帧，存入队列
运动检测线程：从队列取帧进行检测，触发录制后通知写入线程
视频写入线程：后台处理帧写入，不阻塞主线程

使用queue.Queue实现线程间安全通信，避免deque的线程安全问题。

5. 系统层面优化

关闭Ubuntu桌面环境，切换到多用户模式，减少系统资源占用

将CPU调度器设为性能模式：

echo performance | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

升级EasyPySpin和Spinnaker到最新版本，修复潜在性能问题

内容的提问来源于stack exchange，提问作者crookedegg

Blackfly S相机1440x1080分辨率帧率瓶颈排查求助

设备与任务概述 #

核心问题 #

系统负载 #

已尝试方案 #

软硬件配置 #

当前运动检测脚本 #

瓶颈排查与优化建议 #

1. 相机配置与数据传输优化 #

2. 运动检测算法优化 #

3. 缓冲区与视频写入优化 #

4. 线程架构优化 #

5. 系统层面优化 #