使用GStreamer的appsrc解码H264帧时遇到问题

管道配置 #

搭建的GStreamer管道如下：

appsrc name=mysource ! video/x-h264, alignment=au, stream-format=byte-stream ! h264parse ! video/x-h264, alignment=au, stream-format=byte-stream ! omxh264dec ! video/x-raw, format=NV16_10LE32, width=1920, height=1080, framerate=30/1 ! filesink location=ball.raw

帧推送代码 #

逐帧读取带大小信息的H264文件，通过以下函数将帧推送到appsrc：

void VDRE2GStreamerDecodingDisplaySinkTask::PushBufferToGStreamer(uint8_t* encoded_frame, uint32_t frame_size)
{
    GstFlowReturn ret;
    GstMapInfo map;
    GstBuffer *buffer = gst_buffer_new_and_alloc(frame_size);
    gst_buffer_map(buffer, &map, GST_MAP_WRITE);
    map.data = encoded_frame;
    gst_buffer_unmap(buffer, &map);

    /* Push the buffer into the appsrc */
    g_signal_emit_by_name(m_appsrc, "push-buffer", buffer, &ret);

    /* Free the buffer now that we are done with it */
    gst_buffer_unref(buffer);

    if (ret != GST_FLOW_OK) {
        g_print("PushBufferToGStreamer: Error!\n");
    }
}

错误日志 #

执行管道并设置GST_DEBUG=4时，出现大量h264parse报错：

Decode Thread: Frame to Process : 1 :s 246
Decode Thread: Frame to Process : 2 :s 232
Decode Thread: Frame to Process : 3 :s 854
Decode Thread: Frame to Process : 4 :s 952
Decode Thread: Frame to Process : 5 :s 350
Decode Thread: Frame to Process : 6 :s 1038
Decode Thread: Frame to Process : 7 :s 1447
Decode Thread: Frame to Process : 8 :s 1316
Decode Thread: Frame to Process : 9 :s 1297
Decode Thread: Frame to Process : 10 :s 1205
0:00:02.370465675  6548 0xffff8c01ff60 WARN               h264parse gsth264parse.c:1562:gst_h264_parse_handle_frame:<myparser> broken/invalid nal Type: 0 Unknown, Size: 29 will be dropped
0:00:02.370549896  6548 0xffff8c01ff60 WARN               h264parse gsth264parse.c:1562:gst_h264_parse_handle_frame:<myparser> broken/invalid nal Type: 0 Unknown, Size: 1140 will be dropped
Decode Thread: Frame to Process : 11 :s 1222
Decode Thread: Frame to Process : 12 :s 1189
Decode Thread: Frame to Process : 13 :s 1094
Decode Thread: Frame to Process : 14 :s 930
Decode Thread: Frame to Process : 15 :s 1000
Decode Thread: Frame to Process : 16 :s 888
Decode Thread: Frame to Process : 17 :s 901
Decode Thread: Frame to Process : 18 :s 788
Decode Thread: Frame to Process : 19 :s 833
Decode Thread: Frame to Process : 20 :s 764
Decode Thread: Frame to Process : 21 :s 727
Decode Thread: Frame to Process : 22 :s 616
Decode Thread: Frame to Process : 23 :s 728
Decode Thread: Frame to Process : 24 :s 622
Decode Thread: Frame to Process : 25 :s 609
Decode Thread: Frame to Process : 26 :s 657
Decode Thread: Frame to Process : 27 :s 651
Decode Thread: Frame to Process : 28 :s 621
Decode Thread: Frame to Process : 29 :s 534
Decode Thread: Frame to Process : 30 :s 2989
0:00:03.033182378  6548 0xffff8c01ff60 WARN               h264parse gsth264parse.c:1562:gst_h264_parse_handle_frame:<myparser> broken/invalid nal Type: 0 Unknown, Size: 29 will be dropped
0:00:03.033258709  6548 0xffff8c01ff60 WARN               h264parse gsth264parse.c:1562:gst_h264_parse_handle_frame:<myparser> broken/invalid nal Type: 0 Unknown, Size: 117 will be dropped
0:00:03.033309439  6548 0xffff8c01ff60 WARN               h264parse gsth264parse.c:1562:gst_h264_parse_handle_frame:<myparser> broken/invalid nal Type: 0 Unknown, Size: 29 will be dropped
0:00:03.033369800  6548 0xffff8c01ff60 WARN               h264parse gsth264parse.c:1562:gst_h264_parse_handle_frame:<myparser> broken/invalid nal Type: 0 Unknown, Size: 1468 will be dropped
Decode Thread: Frame to Process : 31 :s 411
Decode Thread: Frame to Process : 32 :s 465
Decode Thread: Frame to Process : 33 :s 514
Decode Thread: Frame to Process : 34 :s 520
Decode Thread: Frame to Process : 35 :s 519
Decode Thread: Frame to Process : 36 :s 476
Decode Thread: Frame to Process : 37 :s 448
Decode Thread: Frame to Process : 38 :s 544
Decode Thread: Frame to Process : 39 :s 523
Decode Thread: Frame to Process : 40 :s 454
Decode Thread: Frame to Process : 41 :s 561
Decode Thread: Frame to Process : 42 :s 409
Decode Thread: Frame to Process : 43 :s 506
Decode Thread: Frame to Process : 44 :s 330
Decode Thread: Frame to Process : 45 :s 2180
0:00:03.530095934  6548 0xffff8c01ff60 WARN               h264parse gsth264parse.c:1562:gst_h264_parse_handle_frame:<myparser> broken/invalid nal Type: 0 Unknown, Size: 29 will be dropped
0:00:03.530192075  6548 0xffff8c01ff60 WARN               h264parse gsth264parse.c:1562:gst_h264_parse_handle_frame:<myparser> broken/invalid nal Type: 0 Unknown, Size: 117 will be dropped
0:00:03.530251666  654

编码数据验证 #

分析前5帧的十六进制数据，确认包含有效NAL单元：

Encoded data values in hexadecimal: 0, size: 3241
00 00 00 01 27 7A 00 28 B6 D0 0C E8 07 80 22 7E 27 01 6A 02 04 04 80 00 00 03 00 80 00 00 1E 19 A8 00 3D 09 00 01 6E 37 FF FE 05 00 00 00 01 28 EE 3C

Encoded data values in hexadecimal: 1, size: 246
00 00 00 01 21 E2 01 08 42 DF 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 09 36 B5 84 05 B0 C9 8C DA C9 6F 9D F0 00

Encoded data values in hexadecimal: 2, size: 232
00 00 00 01 21 E4 02 08 42 DF 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 16 D0 00 00 01 21 00 1F E3 90 08 21

Encoded data values in hexadecimal: 3, size: 854
00 00 00 01 21 E6 03 08 8B 7F 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 06 2C 00 00 01 21 00 1F E3 98 0C 22

Encoded data values in hexadecimal: 4, size: 952
00 00 00 01 21 E8 04 0E DF 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 97 80 00 00 01 21 00 1F E3 A0 10 3B

对比测试 #

用filesrc做相同流程测试，管道如下：

gst-launch-1.0 -e filesrc location=ball2.h264 ! video/x-h264, alignment=au, width=1920, height=1080, framerate=30/1 ! h264parse ! omxh264dec ! video/x-raw, format=NV16_10LE32, width=1920, height=1080, framerate=30/1 ! filesink location=ball_gst.raw

该测试无报错，生成的raw文件正常；但appsrc版本的raw文件始终为空，推测未完成解码。已通过identity dump=true确认推送的字节与filesrc读取的完全一致，仍未定位问题。

问题解决方向 #

1. 修正缓冲区拷贝逻辑：当前代码直接修改map.data指向外部指针，未将encoded_frame的数据实际拷贝到GstBuffer内存中。需替换为memcpy(map.data, encoded_frame, frame_size)完成数据拷贝。
2. 补全appsrc属性配置：确认appsrc是否设置了format=GST_FORMAT_TIME，推送完所有帧后需发送EOS信号，让管道完成收尾处理。
3. 简化caps配置：管道中h264parse前后重复设置了相同的caps，可去掉h264parse之后的caps，避免潜在冲突。
4. 检查线程同步：确认推送操作是否在正确的GStreamer线程上下文执行，避免线程冲突导致缓冲区处理异常。

内容的提问来源于stack exchange，提问作者isra60