Python中无法将OpenCV摄像头帧（Numpy数组）管道传给FFmpeg子进程

我尝试用OpenCV捕获摄像头视频流，将原始帧通过管道传入FFmpeg子进程应用3D .cube LUT，再把处理后的帧传回OpenCV用cv2.imshow显示，但代码卡在读取FFmpeg输出的raw_frame = lut_process.stdout.read(width * height * 3)环节，"read"从未打印，FFmpeg卡在0fps，无法显示画面。

我的代码 #

import cv2
import subprocess as sp
import numpy as np

lut_cmd = [
            'ffmpeg', '-f', 'rawvideo', '-pixel_format', 'bgr24', '-s', '1280x720', '-framerate', '30', '-i', '-', '-an', '-vf',
            'lut3d=file=lut/luts/lut.cube', '-f', 'rawvideo', 'pipe:1'
        ]

lut_process = sp.Popen(lut_cmd, stdin=sp.PIPE, stdout=sp.PIPE)

width = 1280
height = 720

video_capture = cv2.VideoCapture(0)

while True:
    ret, frame = video_capture.read()

    if not ret:
        break
 
    # Write raw video frame to input stream of ffmpeg sub-process.
    lut_process.stdin.write(frame.tobytes())
    lut_process.stdin.flush()
    print("flushed")

    # Read the processed frame from the ffmpeg subprocess
    raw_frame = lut_process.stdout.read(width * height * 3)
    print("read")
    frame = np.frombuffer(raw_frame, dtype=np.uint8).reshape(height, width, 3)

    cv2.imshow('Video', frame)

    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

lut_process.terminate()
video_capture.release()

cv2.destroyAllWindows()

运行现象 #

flushed
Input #0, rawvideo, from 'fd:':
  Duration: N/A, start: 0.000000, bitrate: 663552 kb/s
  Stream #0:0: Video: rawvideo (BGR[24] / 0x18524742), bgr24, 1280x720, 663552 kb/s, 30 tbr, 30 tbn
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> rawvideo (native))
Output #0, rawvideo, to 'pipe:1':
  Metadata:
    encoder         : Lavf60.3.100
  Stream #0:0: Video: rawvideo (BGR[24] / 0x18524742), bgr24(progressive), 1280x720, q=2-31, 663552 kb/s, 30 fps, 30 tbn
    Metadata:
      encoder         : Lavc60.3.100 rawvideo
frame=    0 fps=0.0 q=0.0 size=       0kB time=-577014:32:22.77 bitrate=  -0.0kbits/s speed=N/A  

已尝试的方法 #

• 将frame.tobytes()改为frame.tostring()，结果一致
• 在首次写入FFmpeg前添加3秒延迟，问题依旧
• 确认摄像头正常，输出为1280x720 30fps
• 单独用OpenCV显示摄像头流、直接用FFmpeg读取摄像头并通过管道输出给OpenCV显示均成功

环境 #

macOS 12.6、OpenCV 4.7.0、FFmpeg 6.0、Python 3.10.11、Visual Studio Code

问题根源 #

FFmpeg处理管道输入时默认会进行缓冲，不会立即输出处理后的帧，导致Python进程阻塞在stdout.read()等待数据；另外FFmpeg的日志输出（stderr）若未处理，缓冲区满后也会阻塞进程。

修改后的代码 #

import cv2
import subprocess as sp
import numpy as np

# 添加缓冲控制参数，重定向stderr避免阻塞
lut_cmd = [
    'ffmpeg',
    '-fflags', 'nobuffer',
    '-flags', 'low_delay',
    '-f', 'rawvideo',
    '-pixel_format', 'bgr24',
    '-s', '1280x720',
    '-framerate', '30',
    '-i', '-',
    '-an',
    '-vf', 'lut3d=file=lut/luts/lut.cube',
    '-f', 'rawvideo',
    '-pixel_format', 'bgr24',  # 显式指定输出格式，避免不匹配
    'pipe:1'
]

# 重定向stderr到DEVNULL，防止日志缓冲区满阻塞进程
lut_process = sp.Popen(lut_cmd, stdin=sp.PIPE, stdout=sp.PIPE, stderr=sp.DEVNULL)

width = 1280
height = 720

video_capture = cv2.VideoCapture(0)
# 强制设置摄像头参数，确保和FFmpeg输入参数完全匹配
video_capture.set(cv2.CAP_PROP_FRAME_WIDTH, width)
video_capture.set(cv2.CAP_PROP_FRAME_HEIGHT, height)
video_capture.set(cv2.CAP_PROP_FPS, 30)

while True:
    ret, frame = video_capture.read()

    if not ret:
        break
    
    # 确保帧尺寸符合要求，不符合则缩放
    if frame.shape != (height, width, 3):
        frame = cv2.resize(frame, (width, height))
 
    # 写入帧到FFmpeg，捕获管道异常
    try:
        lut_process.stdin.write(frame.tobytes())
        lut_process.stdin.flush()
    except BrokenPipeError:
        print("FFmpeg进程异常退出")
        break

    # 读取处理后的帧，判断是否读取成功
    raw_frame = lut_process.stdout.read(width * height * 3)
    if not raw_frame:
        print("未读取到帧，FFmpeg可能已退出")
        break
    
    frame = np.frombuffer(raw_frame, dtype=np.uint8).reshape(height, width, 3)

    cv2.imshow('Video', frame)

    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# 优化进程退出逻辑，避免僵尸进程
lut_process.stdin.close()
lut_process.wait()
video_capture.release()
cv2.destroyAllWindows()

关键修改点说明 #

1. 添加FFmpeg缓冲控制参数：
   • -fflags nobuffer：禁用输入缓冲，让FFmpeg立即处理传入的帧
   • -flags low_delay：启用低延迟模式，减少输出端的缓冲
2. 重定向stderr：将FFmpeg的日志输出导向sp.DEVNULL，避免日志缓冲区填满导致进程阻塞
3. 显式指定输出像素格式：确保FFmpeg输出的帧格式和OpenCV预期的一致，避免格式不匹配导致读取失败
4. 强制设置摄像头参数：通过video_capture.set()固定摄像头的分辨率和帧率，保证传入FFmpeg的帧参数完全符合要求
5. 优化进程退出逻辑：先关闭stdin，再调用wait()等待FFmpeg正常结束，避免产生僵尸进程

内容的提问来源于stack exchange，提问作者Kevin