Orange Pi5+OV13855摄像头OpenCV采集帧率仅10FPS求助

环境：Orange Pi5、OV13855 MIPI摄像头模块，Ubuntu 22.04、OpenCV 4.5.4、GStreamer 1.20.3

现象：使用GStreamer命令直接采集/dev/video11设备的1920×1080@25FPS视频时一切正常，但通过OpenCV（无论是GStreamer管道方式还是直接调用设备）采集时，帧率仅能达到10FPS。怀疑是OV13855始终输出4224×3136的原始分辨率，导致OpenCV缩放时性能瓶颈，但不确定。

已尝试的操作 #

1. GStreamer正常采集命令

gst-launch-1.0 v4l2src device=/dev/video11 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=25/1 ! videoconvert ! xvimagesink

2. OpenCV通过GStreamer管道采集代码

#include <stdio.h>
#include <memory>
#include <sys/time.h>

#include "opencv2/core/core.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"

int main(int argc, char **argv)
{
    cv::namedWindow("Camera FPS");

    cv::VideoCapture capture("v4l2src device=/dev/video11 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=25/1 ! videoconvert ! appsink drop=true", cv::CAP_GSTREAMER);

    if (!capture.isOpened())
        printf("ERROR: Device is not opened!\n");

    struct timeval time;
    struct timeval time2;

    gettimeofday(&time, nullptr);
    auto startTime = time.tv_sec * 1000 + time.tv_usec / 1000;

    int frames = 0;
    auto beforeTime = startTime;

    while (capture.isOpened())
    {
        cv::Mat img;

        gettimeofday(&time2, nullptr);
        auto startTime2 = time2.tv_usec;

        if (capture.read(img) == false)
            break;

        gettimeofday(&time2, nullptr);
        auto endTime2 = time2.tv_usec;

        cv::imshow("Camera FPS", img);

        if (cv::waitKey(1) == 'q') //Press q to exit
            break;
        frames++;

        if (frames % 30 == 0)
        {
            gettimeofday(&time, nullptr);
            auto currentTime = time.tv_sec * 1000 + time.tv_usec / 1000;
            printf("%f fps/s\n", 30.0 / float(currentTime - beforeTime) * 1000.0);
            printf("%lu us capture time\n", (endTime2-startTime2));
            beforeTime = currentTime;
        }
    }

    while (true)
    {
        cv::Mat img;

        cv::imshow("Camera FPS", img);
        if (cv::waitKey(1) == 'q') //Press q to exit
            break;
        frames++;
    }

    gettimeofday(&time, nullptr);
    auto endTime = time.tv_sec * 1000 + time.tv_usec / 1000;

    printf("Average:\t %f fps/s\n", float(frames) / float(endTime - startTime) * 1000.0);

    return 0;
}

3. OpenCV直接调用设备代码片段

cv::VideoCapture capture(11);
capture.open(11);

capture.set(cv::CAP_PROP_BUFFERSIZE, 1);
capture.set(cv::CAP_PROP_FPS, 25);
capture.set(cv::CAP_PROP_FOURCC, cv::VideoWriter::fourcc('N','V','1','2'));
capture.set(cv::CAP_PROP_FRAME_WIDTH, 1920);
capture.set(cv::CAP_PROP_FRAME_HEIGHT, 1080);

4. v4l2-ctl -d 11 -all查询结果

Driver Info:
        Driver name      : rkisp_v6
        Card type        : rkisp_mainpath
        Bus info         : platform:rkisp0-vir1
        Driver version   : 2.5.1
        Capabilities     : 0x84201000
                Video Capture Multiplanar
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x04201000
                Video Capture Multiplanar
                Streaming
                Extended Pix Format
Media Driver Info:
        Driver name      : rkisp0-vir1
        Model            : rkisp0
        Serial           :
        Bus info         :
        Media version    : 5.10.209
        Hardware revision: 0x00000000 (0)
        Driver version   : 5.10.209
Interface Info:
        ID               : 0x03000007
        Type             : V4L Video
Entity Info:
        ID               : 0x00000006 (6)
        Name             : rkisp_mainpath
        Function         : V4L2 I/O
        Pad 0x01000009   : 0: Sink
          Link 0x0200000a: from remote pad 0x1000004 of entity 'rkisp-isp-subdev' (Unknown V4L2 Sub-Device): Data, Enabled
Priority: 2
Format Video Capture Multiplanar:
        Width/Height      : 1920/1080
        Pixel Format      : 'UYVY' (UYVY 4:2:2)
        Field             : None
        Number of planes  : 1
        Flags             :
        Colorspace        : sRGB
        Transfer Function : Rec. 709
        YCbCr/HSV Encoding: Rec. 709
        Quantization      : Full Range
        Plane 0           :
           Bytes per Line : 3840
           Size Image     : 4147200
Selection Video Capture: crop, Left 0, Top 0, Width 4224, Height 3136, Flags:
Selection Video Capture: crop_bounds, Left 0, Top 0, Width 4224, Height 3136, Flags:
Selection Video Output: crop, Left 0, Top 0, Width 4224, Height 3136, Flags:
Selection Video Output: crop_bounds, Left 0, Top 0, Width 4224, Height 3136, Flags:

解决方案 #

1. 利用RK ISP硬件缩放替代OpenCV软件缩放 #

从v4l2查询结果可知，摄像头链路包含rkisp-isp-subdev（Rockchip ISP子设备），支持硬件裁剪和缩放。GStreamer命令能正常跑满帧率，是因为它触发了ISP的硬件处理；而OpenCV的方式默认可能在用户空间做软件缩放，导致帧率下降。

修改OpenCV的GStreamer管道，让RK ISP完成硬件格式转换和缩放：

cv::VideoCapture capture(
    "v4l2src device=/dev/video11 ! "
    "video/x-raw,format=UYVY,width=4224,height=3136,framerate=25/1 ! "
    "rkispv6proc ! "
    "video/x-raw,format=NV12,width=1920,height=1080 ! "
    "videoconvert ! video/x-raw,format=BGR ! "
    "appsink drop=true", 
    cv::CAP_GSTREAMER
);

说明：

• 先读取摄像头原始分辨率4224×3136的UYVY格式（与v4l2查询的当前格式一致）
• 通过rkispv6proc进行硬件缩放和格式转换，输出1920×1080的NV12
• 最后转成OpenCV兼容的BGR格式，避免OpenCV做额外的软件处理

2. 优化OpenCV直接调用的参数设置 #

直接调用V4L2时，需确保参数正确匹配硬件能力，并启用硬件加速：

cv::VideoCapture capture(11, cv::CAP_V4L2); // 明确指定V4L2后端
if (!capture.isOpened()) {
    printf("ERROR: Device is not opened!\n");
    return -1;
}

// 先设置格式，再设置分辨率和帧率
capture.set(cv::CAP_PROP_FOURCC, cv::VideoWriter::fourcc('U','Y','V','Y')); // 匹配v4l2查询的格式
capture.set(cv::CAP_PROP_FRAME_WIDTH, 4224);
capture.set(cv::CAP_PROP_FRAME_HEIGHT, 3136);
capture.set(cv::CAP_PROP_FPS, 25);
capture.set(cv::CAP_PROP_BUFFERSIZE, 2); // 缓冲区设为2，减少丢帧概率

cv::Mat img, resized;
while (capture.read(img)) {
    // 使用硬件加速的resize（需OpenCV编译时启用Rockchip硬件支持）
    cv::resize(img, resized, cv::Size(1920, 1080), 0, 0, cv::INTER_LINEAR_EXACT);
    cv::imshow("Camera FPS", resized);
    if (cv::waitKey(1) == 'q') break;
}

3. 修复帧率统计的错误代码 #

原代码中循环结束后的无意义while循环会导致平均帧率计算错误，需删除：

// 删除这段无效代码
while (true)
{
    cv::Mat img;

    cv::imshow("Camera FPS", img);
    if (cv::waitKey(1) == 'q') //Press q to exit
        break;
    frames++;
}

4. 验证硬件加速能力 #

用以下命令查看ISP子设备的支持格式和缩放能力：

v4l2-ctl -d /dev/v4l-subdev0 --list-formats-ext # 替换0为实际ISP子设备编号

确认ISP支持从4224×3136缩放到1920×1080的硬件转换。

内容的提问来源于stack exchange，提问作者TeachmeDaddy