C++中使用FFmpeg流传输OpenCV Mat的问题求助

Hey there! Let's work through this FFmpeg + OpenCV streaming issue step by step—since you're new to this, I'll break everything down clearly so you can follow along.

First, let's diagnose your existing problems #

• Why raw output gave you noise: When you pipe raw pixel data to FFmpeg, it has no idea what format that data is in (pixel order, resolution, frame rate, etc.). If you don't explicitly tell FFmpeg these details, it'll guess wrong, resulting in garbage/noise. Also, keep in mind: OpenCV defaults to BGR format for images—if you said your composedImage is RGB, double-check you actually converted it with cvtColor(composedImage, composedImage, COLOR_BGR2RGB); first!
• Why -f jpeg_pipe threw errors: The jpeg_pipe format expects pre-encoded JPEG data as input, but you were sending raw pixel values directly. FFmpeg couldn't parse the raw data as JPEG, hence the error.

Fix 1: Stream raw pixels correctly (no encoding step in OpenCV) #

This is the most efficient path for real-time streaming. Here's what you need to do:

1. Set up the FFmpeg pipe command

Launch FFmpeg as a subprocess with parameters that match your OpenCV image specs. For example, if your composedImage is 1920x1080, RGB24 format, 30fps, and you want to stream to a local UDP address:

ffmpeg -f rawvideo -pix_fmt rgb24 -s 1920x1080 -r 30 -i - -c:v libx264 -preset ultrafast -tune zerolatency -f mpegts udp://192.168.1.100:1234

Let's break down the critical flags:

• -f rawvideo: Tell FFmpeg input is unencoded raw pixels
• -pix_fmt rgb24: Match this to your Mat's pixel format (RGB, 3 bytes per pixel)
• -s 1920x1080: Exact resolution of your composedImage (cols x rows)
• -r 30: Input frame rate
• -i -: Read input from standard input (your pipe)

2. Output OpenCV pixels properly to the pipe

Don't use std::cout << composedImage;—that doesn't output the raw pixel buffer correctly (especially if the Mat has row-alignment padding). Instead, write the raw data directly:

// Verify your Mat is RGB24 (3 channels, 8 bits per channel)
assert(composedImage.type() == CV_8UC3);

// Write raw pixel data to stdout
if (composedImage.isContinuous()) {
    // If the Mat is stored as a single continuous buffer
    std::cout.write(reinterpret_cast<const char*>(composedImage.data), composedImage.total() * composedImage.elemSize());
} else {
    // Handle non-continuous Mats (e.g., cropped ROIs)
    for (int i = 0; i < composedImage.rows; ++i) {
        std::cout.write(reinterpret_cast<const char*>(composedImage.ptr(i)), composedImage.cols * composedImage.elemSize());
    }
}

// Flush the buffer to ensure data sends immediately to FFmpeg
std::cout.flush();

Fix 2: Use jpeg_pipe correctly (encode to JPEG first in OpenCV) #

If you want to use the jpeg_pipe format, you need to encode your Mat to JPEG before sending it to FFmpeg:

1. Encode the Mat to JPEG in OpenCV

std::vector<uchar> jpeg_buffer;
std::vector<int> jpeg_params = {IMWRITE_JPEG_QUALITY, 90}; // Adjust quality as needed

// Encode the image to JPEG
imencode(".jpg", composedImage, jpeg_buffer, jpeg_params);

// Send the encoded JPEG data to stdout
std::cout.write(reinterpret_cast<const char*>(jpeg_buffer.data()), jpeg_buffer.size());
std::cout.flush();

2. Match the FFmpeg command to jpeg_pipe

ffmpeg -f jpeg_pipe -i - -c:v libx264 -preset ultrafast -tune zerolatency -f mpegts udp://192.168.1.100:1234

Now FFmpeg knows to expect JPEG-encoded data from the pipe, so no more errors.

Quick troubleshooting tips #

• Double-check that your FFmpeg input parameters (resolution, pixel format) exactly match your OpenCV Mat. Mismatches here are the #1 cause of noise/garbage frames.
• To receive the stream on another device in your LAN, use ffplay udp://192.168.1.100:1234 (match the UDP address/port to your output command).
• Verify FFmpeg supports your chosen pixel format with ffmpeg -pix_fmts and encoder with ffmpeg -encoders.

内容的提问来源于stack exchange，提问作者cristian b