QT应用中基于FFmpeg的AVFrame转QImage像素格式转换问题排查与优化

Hey there! Let's break down your frame conversion issue and fix this step by step.

Why Your Second Approach Breaks for Some Videos #

The root cause of the weird desaturated/offset images is QImage's memory alignment requirement.

When you create a QImage::Format_RGB888, Qt automatically aligns each scanline to a 4-byte boundary for performance reasons. This means the actual bytes per line (returned by image.bytesPerLine()) isn't always exactly 3 * width. For example:

• If your frame width is 1001, 3*1001 = 3003 (not divisible by 4)
• Qt will round this up to 3004 bytes per line

In your code, you hardcode rgb_linesizes[0] = 3*pFrame->width, which doesn't match Qt's aligned linesize. When sws_scale writes data using this incorrect value, each line gets shifted relative to the next, causing the visual glitches you see.

Fixed Implementation (No Leaks, No Glitches, Optimized) #

Here's a revised version that addresses alignment, reuses FFmpeg resources for speed, and avoids memory issues:

// Store this as a member variable in your class to reuse the conversion context
SwsContext* m_swsContext = nullptr;
int m_lastSrcWidth = 0;
int m_lastSrcHeight = 0;
AVPixelFormat m_lastSrcFormat = AV_PIX_FMT_NONE;

QImage convertFFmpegFrameToQImage(AVFrame* pFrame) {
    if (!pFrame) {
        qWarning() << "Null frame passed for conversion";
        return QImage();
    }

    // Recreate the conversion context only if resolution/format changes
    if (!m_swsContext || 
        m_lastSrcWidth != pFrame->width || 
        m_lastSrcHeight != pFrame->height || 
        m_lastSrcFormat != static_cast<AVPixelFormat>(pFrame->format)) {
        
        // Clean up old context first
        if (m_swsContext) {
            sws_freeContext(m_swsContext);
            m_swsContext = nullptr;
        }

        // Update tracking variables
        m_lastSrcWidth = pFrame->width;
        m_lastSrcHeight = pFrame->height;
        m_lastSrcFormat = static_cast<AVPixelFormat>(pFrame->format);

        // Create new context - use SWS_FAST_BILINEAR for speed, SWS_BICUBIC for better quality
        m_swsContext = sws_getContext(
            pFrame->width, pFrame->height, m_lastSrcFormat,
            pFrame->width, pFrame->height, AV_PIX_FMT_RGB24,
            SWS_FAST_BILINEAR, nullptr, nullptr, nullptr
        );
    }

    if (!m_swsContext) {
        qWarning() << "Failed to initialize SwsContext";
        return QImage();
    }

    // Create QImage with correct alignment
    QImage outputImage(pFrame->width, pFrame->height, QImage::Format_RGB888);
    int dstLinesize = outputImage.bytesPerLine();
    uint8_t* dstData[] = { outputImage.bits() };

    // Perform the color space conversion directly into QImage's memory
    sws_scale(
        m_swsContext,
        pFrame->data, pFrame->linesize,
        0, pFrame->height,
        dstData, &dstLinesize
    );

    return outputImage;
}

// Don't forget to clean up in your class destructor
~YourVideoProcessingClass() {
    if (m_swsContext) {
        sws_freeContext(m_swsContext);
        m_swsContext = nullptr;
    }
}

Key Improvements #

1. Alignment Fix: Uses outputImage.bytesPerLine() instead of hardcoding the linesize, ensuring we match Qt's memory layout exactly.
2. Speed Optimization: Reuses the SwsContext instead of creating/destroying it for every frame. This cuts down on expensive initialization overhead when processing thousands of frames.
3. Safe Memory Management: Writes directly into QImage's internal buffer, so there's no shared memory between FFmpeg and Qt. No leaks, no dangling pointers, and QImage retains full ownership of its data.
4. Flexible Format Handling: Automatically adapts to any input pixel format (YUV, RGB, etc.) that FFmpeg supports.

Bonus: Fixing Your First Approach (If You Ever Need It) #

If you wanted to revisit the "convert to intermediate frame" approach, replace the deprecated avpicture_alloc with modern FFmpeg functions:

// Allocate a frame for RGB conversion
AVFrame* rgbFrame = av_frame_alloc();
rgbFrame->format = AV_PIX_FMT_RGB24;
rgbFrame->width = pFrame->width;
rgbFrame->height = pFrame->height;
av_frame_get_buffer(rgbFrame, 0); // Allocate memory for the frame

// Convert to RGB
sws_scale(m_swsContext, pFrame->data, pFrame->linesize, 0, pFrame->height, rgbFrame->data, rgbFrame->linesize);

// Create QImage by copying the data (so FFmpeg can free the frame without breaking QImage)
QImage image(rgbFrame->data[0], rgbFrame->width, rgbFrame->height, rgbFrame->linesize[0], QImage::Format_RGB888);
QImage copiedImage = image.copy(); // Make a deep copy

// Clean up
av_frame_free(&rgbFrame);

This avoids memory leaks and QImage corruption by copying the data instead of sharing pointers.

内容的提问来源于stack exchange，提问作者Théophane