NVIDIA Jetson平台OpenCV/GStreamer无法获取MIPI相机帧

诊断步骤 #

• 验证GStreamer管道有效性：先通过命令行测试单相机的GStreamer流是否正常，排除硬件/驱动问题：

  gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! video/x-raw(memory:NVMM),width=500,height=320,framerate=30/1,format=NV12 ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw,format=BGR ! autovideosink
  

  如果能正常显示画面，说明相机硬件和GStreamer基础配置没问题。

• 检查OpenCV编译配置：运行以下代码确认OpenCV是否支持GStreamer和NVIDIA硬件加速：

  import cv2
  print(cv2.getBuildInformation())
  

  查找GStreamer项，确保显示为YES，同时确认NVIDIA CUDA相关组件已启用。

• 确认Sensor ID正确性：Jetson上的MIPI相机Sensor ID可能不是连续的0-3，通过以下命令查看已识别的相机设备：

  ls /dev/video*
  

  或者运行sudo jetson-io.py查看相机配置，确保代码中使用的sensor-id与实际匹配。

代码修正方案 #

核心问题是GStreamer管道未将视频格式转换为OpenCV兼容的BGR格式，同时缺少相机初始化状态检查。以下是修正后的代码：

import cv2
import numpy as np
import time

print(cv2.__version__)

# 定义相机GStreamer管道：添加格式转换为BGR，适配OpenCV
def get_cam_pipeline(sensor_id, width=500, height=320, fps=30):
    return (
        f'nvarguscamerasrc sensor-id={sensor_id} ! '
        f'video/x-raw(memory:NVMM), width=(int){width}, height=(int){height}, framerate=(fraction){fps}/1, format=(string)NV12 ! '
        'nvvidconv ! video/x-raw, format=(string)BGRx ! '
        'videoconvert ! video/x-raw, format=(string)BGR ! '
        'appsink'
    )

# 初始化相机并检查状态
cam1 = cv2.VideoCapture(get_cam_pipeline(0), cv2.CAP_GSTREAMER)
cam2 = cv2.VideoCapture(get_cam_pipeline(1), cv2.CAP_GSTREAMER)
cam3 = cv2.VideoCapture(get_cam_pipeline(2), cv2.CAP_GSTREAMER)
cam4 = cv2.VideoCapture(get_cam_pipeline(3), cv2.CAP_GSTREAMER)

cameras = [cam1, cam2, cam3, cam4]
for idx, cam in enumerate(cameras):
    if not cam.isOpened():
        print(f"Failed to open camera {idx}")
        exit(1)

startTime = time.time()

while True:
    frames = []
    ret_flags = []
    # 逐个读取相机帧，记录每个相机的读取状态
    for cam in cameras:
        ret, frame = cam.read()
        ret_flags.append(ret)
        frames.append(frame)
    
    # 检查所有相机是否成功读取帧
    if not all(ret_flags):
        print(f"Error: Camera {ret_flags.index(False)} failed to read frame")
        continue
    
    # 检查帧是否有效
    if any(frame is None or frame.size == 0 for frame in frames):
        print("Error: One or more frames is invalid")
        continue
    
    # 拼接帧
    frame_combined = np.hstack(frames)
    
    cv2.imshow('Combo', frame_combined)
    cv2.moveWindow('Combo', 0, 0)
    
    if cv2.waitKey(1) == ord('q'):
        break

# 释放资源
for cam in cameras:
    cam.release()
cv2.destroyAllWindows()

关键修正点 #

• 格式转换：在GStreamer管道中添加nvvidconv ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR，将NV12格式转换为OpenCV支持的BGR格式。
• 显式指定CAP_GSTREAMER：初始化VideoCapture时添加cv2.CAP_GSTREAMER参数，确保OpenCV使用GStreamer后端。
• 相机初始化检查：添加isOpened()判断，提前发现相机打开失败的问题。
• 逐帧读取状态检查：分别记录每个相机的读取状态，便于定位具体哪个相机出问题。

内容的提问来源于stack exchange，提问作者Interceptor