基于FFmpeg.AutoGen的C# H264多帧解码至位图问题求助

Hey there! I’ve been where you are—diving into FFmpeg.AutoGen in C# can feel overwhelming at first, especially when you’re not getting any output from your H264 decoding. Let’s break down the key steps you might be missing, and walk through a working implementation that converts your H264 frames to .NET Bitmaps.

First, you need to make sure FFmpeg is set up correctly—this is where a lot of new developers stumble. Here’s the critical initialization code you might have skipped:

// Call this once at your app's startup to register FFmpeg binaries
FFmpegBinariesHelper.RegisterFFmpegBinaries();
ffmpeg.av_register_all();
ffmpeg.avcodec_register_all();
ffmpeg.avformat_network_init();

// Locate the H264 decoder codec
var h264Codec = ffmpeg.avcodec_find_decoder(AVCodecID.AV_CODEC_ID_H264);
if (h264Codec == IntPtr.Zero)
{
    throw new InvalidOperationException("Failed to find H264 decoder—check your FFmpeg binaries.");
}

// Allocate and configure the codec context
var codecContext = ffmpeg.avcodec_alloc_context3(h264Codec);
if (codecContext == IntPtr.Zero)
{
    throw new InvalidOperationException("Could not allocate codec context.");
}

// Set core stream parameters using your existing values
codecContext.pix_fmt = AVPixelFormat.AV_PIX_FMT_YUV420P; // Most H264 streams use this format
codecContext.width = nWidth;
codecContext.height = nHeight;

// Open the codec—this is mandatory before decoding
if (ffmpeg.avcodec_open2(codecContext, h264Codec, IntPtr.Zero) < 0)
{
    throw new InvalidOperationException("Failed to open H264 codec.");
}

Note: If your stream uses a different pixel format (rare for H264), adjust pix_fmt accordingly. YUV420P is the standard.

FFmpeg expects input data wrapped in an AVPacket—you’ll need to package your pVideo pointer and nVideoSize into this structure:

var packet = new AVPacket();
ffmpeg.av_init_packet(packet);

// Assign your raw H264 data to the packet
packet.data = pVideo;
packet.size = nVideoSize;

// Optional but recommended: Set PTS/DTS values if your stream provides them
// This helps with correct frame ordering for P/B frames
// packet.pts = yourFramePts;
// packet.dts = yourFrameDts;

H264 decodes to YUV frames by default, which .NET’s Bitmap doesn’t natively understand. We’ll need to convert it to a RGB format (BGR24, since Bitmaps use BGR under the hood) using FFmpeg’s scaling context.

First, allocate frames for decoding and conversion:

var decodedYuvFrame = ffmpeg.av_frame_alloc();
var convertedRgbFrame = ffmpeg.av_frame_alloc();

// Calculate buffer size for the RGB frame
int rgbBufferSize = ffmpeg.av_image_get_buffer_size(AVPixelFormat.AV_PIX_FMT_BGR24, nWidth, nHeight, 1);
IntPtr rgbBuffer = Marshal.AllocHGlobal(rgbBufferSize);

// Attach the buffer to the RGB frame
ffmpeg.av_image_fill_arrays(
    convertedRgbFrame.data, 
    convertedRgbFrame.linesize, 
    rgbBuffer, 
    AVPixelFormat.AV_PIX_FMT_BGR24, 
    nWidth, nHeight, 1);

// Create a scaling context to convert YUV to RGB
var scalingContext = ffmpeg.sws_getContext(
    nWidth, nHeight, codecContext.pix_fmt,
    nWidth, nHeight, AVPixelFormat.AV_PIX_FMT_BGR24,
    ffmpeg.SWS_BILINEAR, IntPtr.Zero, IntPtr.Zero, IntPtr.Zero);

Now perform the decode and conversion:

// Send the packet to the decoder
int sendResult = ffmpeg.avcodec_send_packet(codecContext, packet);
if (sendResult < 0)
{
    ffmpeg.av_packet_unref(packet);
    return null; // Handle corrupted/invalid packet
}

// Receive the decoded YUV frame
int receiveResult = ffmpeg.avcodec_receive_frame(codecContext, decodedYuvFrame);
if (receiveResult == 0)
{
    // Convert YUV frame to BGR24
    ffmpeg.sws_scale(
        scalingContext,
        decodedYuvFrame.data, decodedYuvFrame.linesize, 0, nHeight,
        convertedRgbFrame.data, convertedRgbFrame.linesize);

    // Create a Bitmap from the RGB buffer
    var outputBitmap = new Bitmap(nWidth, nHeight, PixelFormat.Format24bppRgb);
    var bitmapData = outputBitmap.LockBits(
        new Rectangle(0, 0, nWidth, nHeight),
        ImageLockMode.WriteOnly,
        PixelFormat.Format24bppRgb);

    // Copy the RGB buffer to the Bitmap's memory
    Marshal.Copy(rgbBuffer, bitmapData.Scan0, 0, rgbBufferSize);

    outputBitmap.UnlockBits(bitmapData);

    // Clean up temporary resources
    ffmpeg.av_frame_unref(decodedYuvFrame);
    ffmpeg.av_packet_unref(packet);

    return outputBitmap;
}
else if (receiveResult == ffmpeg.AVERROR(ffmpeg.EAGAIN))
{
    // Decoder needs more data—send the next frame packet
    ffmpeg.av_packet_unref(packet);
    return null;
}
else
{
    ffmpeg.av_packet_unref(packet);
    throw new InvalidOperationException($"Decoding failed with error code: {receiveResult}");
}

FFmpeg doesn’t manage memory automatically—be sure to free all allocated resources when you’re done:

ffmpeg.sws_freeContext(scalingContext);
Marshal.FreeHGlobal(rgbBuffer);
ffmpeg.av_frame_free(convertedRgbFrame);
ffmpeg.av_frame_free(decodedYuvFrame);
ffmpeg.avcodec_close(codecContext);
ffmpeg.av_free(codecContext);
ffmpeg.avformat_network_deinit();

• Missing Codec Context Configuration: If you didn’t set width, height, and pix_fmt, the decoder has no idea how to process your data.
• Ignoring EAGAIN: This return code means the decoder needs more packets to produce a frame—don’t treat it as an error.
• Skipping Pixel Format Conversion: H264 doesn’t decode to RGB directly—you must convert YUV to a Bitmap-compatible format.
• Forgetting to Unref Packets/Frames: Failing to call av_packet_unref or av_frame_unref will cause memory leaks and unpredictable behavior.
• Not Sending I-Frames First: Decoders require an I-frame to start decoding P/B frames—ensure your first packet is an I-frame.

内容的提问来源于stack exchange，提问作者Jasim Khan Afridi