Qt C++结合GStreamer开发如何将GstSample转换为QImage

你当前代码输出乱码的核心原因有两个：

• pipeline未配置颜色空间转换和格式约束，appsink拿到的帧格式、宽高都是默认值，和你硬编码的QImage参数完全不匹配
• 没有从GstSample的配置信息中提取实际的帧参数，直接写死宽高、像素格式，肯定无法正确解析数据

第一步：修改pipeline配置，统一输出QImage兼容的格式 #

你需要在源和appsink之间增加videoconvert元素做颜色格式转换，同时给appsink设置caps约束，强制输出QImage原生支持的像素格式，避免后续额外做格式映射。
修改后的pipeline初始化代码如下：

/* Initialize GStreamer */
gst_init (NULL, NULL);

/* Create the elements */
GstElement * source = gst_element_factory_make ("videotestsrc", "source");
GstElement * videoconvert = gst_element_factory_make("videoconvert", "convert");
GstElement * sink = gst_element_factory_make ("appsink", "sink");
GstAppSink *appsink = GST_APP_SINK(sink);

/* Create the empty pipeline */
GstElement * pipeline = gst_pipeline_new ("test-pipeline");

if (!pipeline || !source || !videoconvert || !sink) {
    g_printerr ("Not all elements could be created.\n");
    return ;
}

/* Build the pipeline */
gst_bin_add_many (GST_BIN (pipeline), source, videoconvert, sink, NULL);
if (gst_element_link_many (source, videoconvert, sink, NULL) != TRUE) {
    g_printerr ("Elements could not be linked.\n");
    gst_object_unref (pipeline);
    return ;
}

/* 配置appsink输出格式：强制为RGBA8888，和QImage原生格式对应 */
GstCaps *caps = gst_caps_new_simple("video/x-raw",
    "format", G_TYPE_STRING, "RGBA",
    NULL);
g_object_set(appsink, "caps", caps, "drop", TRUE, "max-buffers", 3, NULL);
gst_caps_unref(caps);

/* Modify the source's properties */
g_object_set (source, "pattern", 0, NULL);

/* Start playing */
GstStateChangeReturn ret = gst_element_set_state (pipeline, GST_STATE_PLAYING);
if (ret == GST_STATE_CHANGE_FAILURE) {
    g_printerr ("Unable to set the pipeline to the playing state.\n");
    gst_object_unref (pipeline);
    return;
}

第二步：从GstSample提取参数生成QImage #

拿到sample后先从其附带的caps信息中读取实际的帧宽高，不要硬编码参数，同时注意资源释放避免内存泄漏：

GstSample *sample = nullptr;
do{
    sample = gst_app_sink_pull_sample(appsink);
    if (!sample) {
        printf("sample is NULL\n");
    }else{
        GstBuffer * buffer = gst_sample_get_buffer(sample);
        GstMapInfo info;
        if(gst_buffer_map(buffer, &info, GST_MAP_READ)){
            // 从sample的配置信息中读取实际帧宽高
            GstCaps *sample_caps = gst_sample_get_caps(sample);
            GstStructure *struc = gst_caps_get_structure(sample_caps, 0);
            int width, height;
            gst_structure_get_int(struc, "width", &width);
            gst_structure_get_int(struc, "height", &height);

            // 生成QImage，格式和我们之前设置的RGBA对应
            // 注意：如果需要长期持有QImage，需要调用.copy()，因为info.data会在buffer释放后失效
            QImage image((const uchar*)info.data, width, height, QImage::Format_RGBA8888);
            // 可按需转换为无alpha通道的RGB格式
            QImage rgbImage = image.convertToFormat(QImage::Format_RGB888);
            rgbImage.save("out.jpg");

            gst_buffer_unmap(buffer, &info);
        }
        gst_sample_unref(sample); // 必须释放sample，否则会出现内存泄漏
    }
}while(sample);

注意事项 #

• 后续对接RTSP源时，把videotestsrc替换为rtspsrc + 解封装、解码链路再接videoconvert即可正常工作
• 拉帧循环不要放在Qt UI主线程执行，建议放到独立工作线程运行，避免阻塞界面

内容的提问来源于stack exchange，提问作者Daniel Lopes