WPF中使用GStreamer udpsrc显示视频失败，寻求技术帮助

命令行执行GStreamer UDP视频收发命令可正常工作，但在WPF代码中存在两个核心问题：

• Gst.Video.Global.IsVideoOverlayPrepareWindowHandleMessage(msg)始终返回False，无法完成窗口句柄绑定
• 通过AppSink拉取Sample时，返回值始终为null

发送端命令 #

gst-launch-1.0.exe autovideosrc ! videoconvert ! vp8enc deadline=1 ! rtpvp8pay pt=96 ! udpsink port=5200 host=<MY-LOCALIP> async=false

接收端命令 #

gst-launch-1.0.exe udpsrc port=5200 caps="application/x-rtp,payload=(int)96" ! rtpjitterbuffer latency=10 ! rtpvp8depay ! avdec_vp8 output-corrupt=false ! videoconvert ! autovideosink

相关C#代码 #

创建Pipeline方法

private void CreatePipeLine()
{
        if(pipeline != null)
        {
            pipeline.SetState(State.Null);
            islive = false;
            return;
        }
    
        pipeline = new Pipeline("pipeline");//"playback");
    
        var udpsrc = Gst.ElementFactory.Make("udpsrc", "source");
        udpsrc["port"] = 5200;
        //udpsrc["caps"] = "application/x-rtp,payload=96";
    
        var jitter = Gst.ElementFactory.Make("rtpjitterbuffer", "jitter");
        jitter["latency"] = 10;
    
        var depay = Gst.ElementFactory.Make("rtpvp8depay", "depay");
        var avdec = Gst.ElementFactory.Make("avdec_vp8", "avdec");
        avdec["output-corrupt"] = false;
    
        var convert = Gst.ElementFactory.Make("videoconvert", "convert");
    
        var capsRTP = Gst.Global.CapsFromString("application/x-rtp,payload=96");
        var mAppSink = new AppSink("appsink");
        mAppSink["emit-signals"] = true;
        mAppSink["caps"] = capsRTP;
        mAppSink.NewSample += OnNewSample;
    
        ink = Gst.ElementFactory.Make("autovideosink", "ink");
    
        pipeline.Add(udpsrc, jitter, depay, avdec, convert, mAppSink); //ink);
    
             
        udpsrc.Link(jitter);
        jitter.Link(depay);
        depay.Link(avdec);
        avdec.Link(convert);
        convert.Link(ink);
    
        Bus bus = pipeline.Bus;
        bus.AddSignalWatch();
        bus.Message += OnBusMessage;
    
        bus.EnableSyncMessageEmission();
        bus.SyncMessage += new SyncMessageHandler(OnBusSynMessage);
    
        /*
        VideoSink = new AppSink(ink.Handle);
        VideoSink.Drop = true;
        VideoSink.MaxLateness = (1000 / 30) * 1000000;
        VideoSink.MaxBuffers = 1;
        VideoSink.Qos = true;
        VideoSink.EnableLastSample = false;
        VideoSink.Caps = Gst.Caps.FromString("video/x-raw,format=RGBA");
    
        pipeline.Bus.EnableSyncMessageEmission();
        pipeline.Bus.AddSignalWatch();
        pipeline.Bus.SyncMessage += OnBusSynMessage;
        pipeline.Bus.Message += OnBusMessage;
        */
    
        pipeline.SetState(State.Null);
        var ret = pipeline.SetState(State.Ready);
}

总线同步消息处理方法

private void OnBusSynMessage(object o, SyncMessageArgs sargs)
{
    Gst.Message msg = sargs.Message;
    if (!Gst.Video.Global.IsVideoOverlayPrepareWindowHandleMessage(msg)) { return;}

    Element src = msg.Src as Element;
    if(src == null) { return;}
    try { src["force-aspect-ration"] = true;}
    catch(PropertyNotFoundException) {  }

    Element overlay = ((Gst.Bin)src).GetByInterface(VideoOverlayAdapter.GType);
    if(overlay == null) 
    {
        System.Diagnostics.Debug.WriteLine($"Overlay is null");
        return;
    }

    _adapter = new VideoOverlayAdapter(overlay.Handle);
    _adapter.WindowHandle = _windowHandle;
    _adapter.HandleEvents(true);
    _isRender = true;

}

Sample拉取代码

var sink = VideoSink;
if(sink == null) { return; }

Sample sample = sink.TryPullSample(5000);
if(sample == null) 
{
    return; 
}

已尝试的解决方法 #

• 调整caps的设置位置
• 使用autovideosink替代自定义Sink

问题仍未解决，寻求帮助。

内容的提问来源于stack exchange，提问作者user20571402