OpenCV-GStreamer后端无法播放gst-launch可正常运行的UDP视频流

环境说明 #

Windows系统，基于GStreamer 1.24.3在Visual Studio 22编译OpenCV-4.9.0源码，已配置必要路径，测试程序可正常播放videotestsrc测试流，但将UDP摄像头的GStreamer管道转为Python代码后无法创建管道，无画面输出。

测试通过的代码 #

import os
os.add_dll_directory("D:\opencv-4.9.0\opencv-4.9.0\build\install\x64\vc17\bin")
os.add_dll_directory("D:\opencv-4.9.0\opencv-4.9.0\build\install\x64\vc17\lib")
os.add_dll_directory("D:\gstreamer\1.0\msvc_x86_64\bin")
import cv2 
cap = cv2.VideoCapture('videotestsrc ! videoconvert ! appsink')
while True:
    ret,frame = cap.read()
    cv2.imshow('',frame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break
cv2.destroyAllWindows()
cap.release() 

可正常运行的gst-launch命令 #

gst-launch-1.0 udpsrc buffer-size=600000 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, width=(string)640, height=(string)480, colorimetry=(string)BT709-2, payload=(int)100" port=50001 ! rtpvrawdepay ! glimagesink

执行失败的Python代码 #

import os
os.add_dll_directory("D:\opencv-4.9.0\opencv-4.9.0\build\install\x64\vc17\bin")
os.add_dll_directory("D:\opencv-4.9.0\opencv-4.9.0\build\install\x64\vc17\lib")
os.add_dll_directory("D:\gstreamer\1.0\msvc_x86_64\bin")
import cv2 
cap = cv2.VideoCapture('udpsrc buffer-size=600000 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, width=(string)640, height=(string)480, colorimetry=(string)BT709-2, payload=(int)100" port=50001 ! rtpvrawdepay ! appsink', cv2.CAP_GSTREAMER)
while True:
    ret,frame = cap.read()
    if ret:
        cv2.imshow('',frame)
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break
cv2.destroyAllWindows()
cap.release()

错误日志 #

[ WARN:0@0.082] global cap_gstreamer.cpp:2839 cv::handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module udpsrc0 reported: Internal data stream error.
[ WARN:0@0.082] global cap_gstreamer.cpp:1698 cv::GStreamerCapture::open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0@0.083] global cap_gstreamer.cpp:1173 cv::GStreamerCapture::isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
[ WARN:0@0.084] global cap.cpp:206 cv::VideoCapture::open VIDEOIO(GSTREAMER): backend is generally available but can't be used to capture by name

解决步骤 #

1. 修复GStreamer管道格式问题 #

原代码中的管道存在两个关键问题：

• port=50001被错误放在caps参数之后，不属于udpsrc的属性范围
• 缺少格式转换环节，rtpvrawdepay输出的原始格式与OpenCV的appsink不兼容

修改后的管道字符串：

udpsrc port=50001 buffer-size=600000 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, width=(string)640, height=(string)480, colorimetry=(string)BT709-2, payload=(int)100" ! rtpvrawdepay ! videoconvert ! appsink sync=false

关键调整：

• 将port=50001移至udpsrc的属性列表中，确保参数归属正确
• 添加videoconvert元素，完成格式转换以匹配appsink的预期格式
• 增加sync=false参数，避免时钟同步问题导致的流中断

2. 验证修改后的管道 #

先在命令行测试修改后的管道是否正常工作：

gst-launch-1.0 udpsrc port=50001 buffer-size=600000 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, width=(string)640, height=(string)480, colorimetry=(string)BT709-2, payload=(int)100" ! rtpvrawdepay ! videoconvert ! glimagesink

3. 更新后的Python代码 #

import os
# 使用原始字符串避免转义问题
os.add_dll_directory(r"D:\opencv-4.9.0\opencv-4.9.0\build\install\x64\vc17\bin")
os.add_dll_directory(r"D:\opencv-4.9.0\opencv-4.9.0\build\install\x64\vc17\lib")
os.add_dll_directory(r"D:\gstreamer\1.0\msvc_x86_64\bin")
import cv2 

# 修复后的GStreamer管道
gst_pipeline = 'udpsrc port=50001 buffer-size=600000 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, width=(string)640, height=(string)480, colorimetry=(string)BT709-2, payload=(int)100" ! rtpvrawdepay ! videoconvert ! appsink sync=false'
cap = cv2.VideoCapture(gst_pipeline, cv2.CAP_GSTREAMER)

# 先检查管道是否成功打开
if not cap.isOpened():
    print("无法初始化视频流管道")
    exit()

while True:
    ret, frame = cap.read()
    if ret:
        cv2.imshow("UDP视频流", frame)
        # 处理退出逻辑
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break
    else:
        print("读取视频帧失败")
        break

# 清理资源
cv2.destroyAllWindows()
cap.release()

4. 额外排查点 #

• 确认UDP端口50001未被其他程序占用（可通过netstat -ano | findstr :50001检查）
• 验证GStreamer插件完整性：执行gst-inspect-1.0 rtpvrawdepay，若输出插件信息则正常
• 检查OpenCV编译配置：运行print(cv2.getBuildInformation())，确认GStreamer项显示为YES

内容的提问来源于stack exchange，提问作者Itqan Ullah