GStreamer配置RTSPServer不影响原流及重连失败问题咨询

需求说明 #

搭建视频流系统，需同时实现两项功能：

• 对输入流进行超低延迟的缩放及计算处理；
• 将原流提供给RTSP客户端（可接受一定延迟，支持多次连接与断开）。

实现方案 #

GStreamer分流管道 #

rtspsrc name=rtspsrc latency=0 location=" + rtspString + " add-reference-timestamp-meta=true ! 
rtph264depay name=depay ! 
tee name=t ! 
h264parse ! 
avdec_h264 ! 
decodebin name=decoder ! 
videoscale qos=false n-threads=2 ! 
videoconvert qos=false n-threads=2 ! 
autovideosink name=autoSink sync=false 
t. ! queue leaky=2 max-size-buffers=10 ! appsink name=appsink sync=false

（autoSink为自定义处理的替代输出端）

RTSP服务器代码 #

// RTSP Server
server = gst_rtsp_server_new();
g_signal_connect(server, "client-connected", G_CALLBACK(client_connected), this);

GstRTSPMountPoints* mounts = gst_rtsp_server_get_mount_points(server);
GstRTSPMediaFactory* factory = gst_rtsp_media_factory_new();
gst_rtsp_media_factory_set_launch(factory, "( appsrc name=appsrc ! rtph264pay config-interval=1 name=pay0 pt=96 )");
GstElement* appsink = gst_bin_get_by_name(GST_BIN(pipeline), "appsink");
g_signal_connect(factory, "media_configure", (GCallback)media_configure, appsink);
gst_rtsp_media_factory_set_stop_on_disconnect(factory, TRUE);
gst_rtsp_media_factory_set_shared(factory, TRUE);
gst_rtsp_mount_points_add_factory(mounts, "/stream", factory);
g_object_unref(mounts);

gst_rtsp_server_attach(server, NULL);

遇到的问题 #

• 首次RTSP客户端连接正常，断开时出现警告：

0:00:18.886077400  8708 0000027C05C6C010 WARN               rtspmedia rtsp-media.c:5377:gst_rtsp_media_set_state: media 0000027C0CC14570 was not prepared
Client disconnected, total clients: 0

• 再次连接时出现如下警告后整个管道停止：

0:00:30.730007700  8708 0000027C05249B70 WARN         rtpjitterbuffer rtpjitterbuffer.c:634:calculate_skew: delta - skew: 0:00:03.506467622 too big, reset skew
0:00:30.735919000  8708 0000027C05249B70 WARN         rtpjitterbuffer rtpjitterbuffer.c:634:calculate_skew: delta - skew: 0:00:01.005564255 too big, reset skew

相关回调处理函数 #

static void
client_connected(GstRTSPServer* server, GstRTSPClient* client, gpointer data)
{
    rtspClientCount++;
    g_signal_connect(client, "closed", G_CALLBACK(client_closed), data);
    g_print("Client connected, total clients: %u\n", rtspClientCount);
}

static void
client_closed(GstRTSPClient* client, gpointer data)
{
    rtspClientCount--;
    g_print("Client disconnected, total clients: %u\n", rtspClientCount);
}

static GstFlowReturn
on_new_sample(GstElement* appsink, gpointer app_src)
{
    if (instance->rtspClientCount == 0 || app_src == NULL)
        return GST_FLOW_OK;

    GstSample* sample;
    GstElement* appsrc = GST_ELEMENT(app_src);
    GstFlowReturn ret;

    // Pull the sample from the appsink
    g_signal_emit_by_name(appsink, "pull-sample", &sample);

    if (sample) {
        // Push the sample to the appsrc in the RTSP pipeline
        ret = gst_app_src_push_sample(GST_APP_SRC(appsrc), sample);
        gst_sample_unref(sample);
        return GST_FLOW_OK;
    }
    return GST_FLOW_ERROR;
}

static void
media_configure(GstRTSPMediaFactory* factory, GstRTSPMedia* media, gpointer app_sink)
{
    /* get the element used for providing the streams of the media */
    GstElement* element = gst_rtsp_media_get_element(media);
    GstElement* appsrc = gst_bin_get_by_name_recurse_up(GST_BIN(element), "appsrc");
    GstElement* appsink = (GstElement*)app_sink;
    g_object_set(G_OBJECT(appsink), "emit-signals", TRUE, NULL);
    g_object_set(G_OBJECT(appsrc), "format", GST_FORMAT_TIME,
                                    "is-live", TRUE, NULL);

    /* install the callback that will be called when a buffer is needed */
    g_signal_connect(appsink, "new-sample", G_CALLBACK(on_new_sample), appsrc);
    g_object_unref(element);
    gst_object_unref(appsrc);
}

咨询问题 #

1. 是否有更优的方案实现上述需求？
2. 为何无法多次重连，是否需要额外的清理或断开处理逻辑？

一、更优实现方案 #

当前方案对原流做了解码再推送的操作完全冗余——原流已是H.264编码的RTSP流，可直接分流转发，无需解码再编码，既节省资源又降低延迟。优化后的实现如下：

优化后的GStreamer管道 #

直接将rtph264depay后的H.264裸流分流，一路走解码处理分支，另一路直接传送给RTSP服务器的appsrc：

rtspsrc name=rtspsrc latency=0 location=" + rtspString + " add-reference-timestamp-meta=true ! 
rtph264depay name=depay ! 
tee name=t ! 
h264parse ! 
avdec_h264 ! 
decodebin name=decoder ! 
videoscale qos=false n-threads=2 ! 
videoconvert qos=false n-threads=2 ! 
autovideosink name=autoSink sync=false 
t. ! queue leaky=2 max-size-buffers=10 ! h264parse ! appsink name=appsink sync=false emit-signals=true

这里给appsink直接添加emit-signals=true，减少代码中的重复设置。

RTSP服务器优化 #

将H.264解析的工作放到服务器端，符合GStreamer组件分工：

gst_rtsp_media_factory_set_launch(factory, "( appsrc name=appsrc ! h264parse ! rtph264pay config-interval=1 name=pay0 pt=96 )");

另外，若要进一步降低处理分支的延迟，可做以下调整：

• 有NVIDIA GPU时，用nvvidconv替代videoscale+videoconvert，硬件加速能大幅降低延迟和CPU占用；
• 若源流可靠，给rtspsrc设置do-retransmission=false，避免重传带来的延迟；
• 保持处理分支的qos=false设置，关闭质量控制以优先低延迟。

二、多次重连失败的原因与修复 #

核心问题分析 #

1. 回调重复绑定：每次客户端连接触发media_configure时，都会给同一个appsink绑定一次new-sample回调，多次绑定后导致回调重复调用，引发流混乱；
2. 媒体状态未清理：gst_rtsp_media_factory_set_stop_on_disconnect(factory, TRUE)会在客户端断开时停止媒体流，但再次连接时，原有的appsrc和回调绑定关系未清理，导致时间戳偏差，触发rtpjitterbuffer警告；
3. 无客户端时的缓存堆积：on_new_sample中客户端数为0时直接返回GST_FLOW_OK，但appsink仍会缓存旧帧，再次连接时推送旧帧，时间戳与当前流不匹配，最终导致管道停止。

修复步骤 #

1. 清理重复回调绑定：在media_configure中绑定新回调前，先断开之前的绑定：

static void
media_configure(GstRTSPMediaFactory* factory, GstRTSPMedia* media, gpointer app_sink)
{
    GstElement* element = gst_rtsp_media_get_element(media);
    GstElement* appsrc = gst_bin_get_by_name_recurse_up(GST_BIN(element), "appsrc");
    GstElement* appsink = (GstElement*)app_sink;
    
    // 断开之前的回调绑定
    g_signal_handlers_disconnect_by_func(appsink, G_CALLBACK(on_new_sample), NULL);
    
    g_object_set(G_OBJECT(appsink), "emit-signals", TRUE, NULL);
    g_object_set(G_OBJECT(appsrc), "format", GST_FORMAT_TIME,
                                    "is-live", TRUE, NULL);

    g_signal_connect(appsink, "new-sample", G_CALLBACK(on_new_sample), appsrc);
    
    gst_rtsp_media_set_state(media, GST_STATE_PLAYING);
    g_object_unref(element);
    gst_object_unref(appsrc);
}

2. 处理无客户端时的缓存：客户端数为0时，拉取sample后直接丢弃，避免appsink缓存旧帧：

static GstFlowReturn
on_new_sample(GstElement* appsink, gpointer app_src)
{
    GstSample* sample;
    g_signal_emit_by_name(appsink, "pull-sample", &sample);

    if (!sample) {
        return GST_FLOW_ERROR;
    }

    if (instance->rtspClientCount == 0 || app_src == NULL) {
        gst_sample_unref(sample);
        return GST_FLOW_OK;
    }

    GstElement* appsrc = GST_ELEMENT(app_src);
    GstFlowReturn ret = gst_app_src_push_sample(GST_APP_SRC(appsrc), sample);
    gst_sample_unref(sample);
    return ret;
}

3. 调整媒体状态管理：将gst_rtsp_media_factory_set_stop_on_disconnect(factory, TRUE)改为FALSE，让媒体流保持运行状态，仅在无客户端时停止推送；同时在client_closed中清理appsrc状态：

// 需提前保存全局的appsrc引用
static GstElement* global_appsrc = NULL;

static void
client_closed(GstRTSPClient* client, gpointer data)
{
    rtspClientCount--;
    g_print("Client disconnected, total clients: %u\n", rtspClientCount);
    
    if (rtspClientCount == 0 && global_appsrc != NULL) {
        gst_app_src_end_of_stream(GST_APP_SRC(global_appsrc));
    }
}

通过以上调整，即可解决多次重连的问题，同时优化后的方案能提升系统性能与延迟表现。

内容的提问来源于stack exchange，提问作者Peter