GStreamer用appsrc/appsink+x264enc编码NV12到H264仅首帧正常的问题

问题背景 #

尝试使用GStreamer的appsrc和appsink组件，通过x264enc将NV12格式的视频帧编码为H264文件。基于jpegenc编码原始帧为JPEG的示例代码修改后可正常工作，但切换为x264enc后，仅首帧能正常解码播放，后续帧使用mplayer播放时出现PPS相关错误。

首帧播放输出 #

[h264 @ 0x7f1ea2ed3600]decoding for stream 0 failed
[lavf] stream 0: video (h264), -vid 0
VIDEO:  [H264]  1920x1080  0bpp  1.000 fps    0.0 kbps ( 0.0 kbyte/s)
==========================================================================
Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
libavcodec version 58.54.100 (external)
Selected video codec: [ffh264] vfm: ffmpeg (FFmpeg H.264)
==========================================================================
Load subtitles in ./
Audio: no sound
Starting playback...
Movie-Aspect is 1.78:1 - prescaling to correct movie aspect.
VO: [vdpau] 1920x1080 => 1920x1080 Planar YV12

错误帧播放输出 #

[h264 @ 0x7f9c2e44f4c0]non-existing PPS 0 referenced
[h264 @ 0x7f9c2e44f4c0]non-existing PPS 0 referenced
[h264 @ 0x7f9c2e44f4c0]decode_slice_header error

[h264 @ 0x7f9c2eece600]decoding for stream 0 failed
[h264 @ 0x7f9c2eece600]Could not find codec parameters for stream 0 (Video: h264, none): unspecified size
Consider increasing the value for the 'analyzeduration' and 'probesize' options
[lavf] stream 0: video (h264), -vid 0
VIDEO:  [H264]  0x0  0bpp  25.000 fps    0.0 kbps ( 0.0 kbyte/s)
==========================================================================
Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
libavcodec version 58.54.100 (external)
Selected video codec: [ffh264] vfm: ffmpeg (FFmpeg H.264)

当前代码片段 #

pipeline = gst_pipeline_new("mypipeline");
appsrc = gst_element_factory_make("appsrc", "mysource");
queue1 = gst_element_factory_make("queue", "myqueue1");
jpegenc = gst_element_factory_make("x264enc", "myenc");
//parser = gst_element_factory_make("h264parse", "parser");
appsink = gst_element_factory_make("appsink", "mysink");


...
g_object_set(G_OBJECT(appsrc), "format", GST_FORMAT_TIME, "is-live", TRUE, "do-timestamp", TRUE, NULL);
g_object_set(G_OBJECT(jpegenc), "tune", 0x00000004, NULL); // 0x00000004 is the ultrafast preset

...
capsappsrc2Jpegenc = gst_caps_new_simple("video/x-raw", "format",
            G_TYPE_STRING, "NV12", "width", G_TYPE_INT, width, "height",
            G_TYPE_INT, height, "framerate", GST_TYPE_FRACTION, 1, 1, 
            "interlace-mode", G_TYPE_STRING, "progressive",
            "colorimetry", G_TYPE_STRING, "bt601",
            NULL);

    // blocksize is important for jpegenc to know how many data to expect from appsrc in a single frame, too
    char szTemp[64];
    sprintf(szTemp, "%d", 3110400);
    g_object_set(G_OBJECT (appsrc), "blocksize", szTemp,
            NULL);

解决方案 #

• 必须启用h264parse元素
  x264enc输出的是原始H.264 NALU流，无统一封装结构，h264parse可整理NALU序列，确保关键帧携带的SPS/PPS信息被解码器正确识别。取消注释代码中的parser初始化，并将其插入到x264enc与appsink之间完成链路拼接。

• 强制x264enc重复输出序列头
  默认x264enc仅在首帧输出SPS/PPS，后续帧依赖解码器缓存，一旦缓存丢失就会报错。通过以下设置让每个关键帧前都携带SPS/PPS：

g_object_set(G_OBJECT(jpegenc), "repeat-sequence-header", TRUE, NULL);

• 修正blocksize参数的设置方式
  blocksize是整数类型参数，无需转为字符串，直接传入数值即可：

g_object_set(G_OBJECT(appsrc), "blocksize", 3110400, NULL);

• 校验时间戳与帧率一致性
  确保appsrc推送的每个buffer都带有正确递增的时间戳，或确认framerate设置与实际输入帧速率匹配，避免x264enc的GOP结构异常。

内容的提问来源于stack exchange，提问作者Abdul Rehman