使用GStreamer与NNStreamer调用VX Delegate失败求助

问题背景 #

使用BoundaryDevices iMX8MP开发板，计划通过TFLite结合VX Delegate在NPU上运行目标检测推理任务。已确认libvx_delegate.so动态库可用，benchmark-model和modelrunner工具均能正常调用NPU，但运行GStreamer流水线时出现推理初始化失败错误。

使用的GStreamer命令 #

gst-launch-1.0 \
    videotestsrc is-live=1 ! \      
    video/x-raw,width=640,height=480 ! \
    tee name=t \    
    t. ! queue name=thread-nn max-size-buffers=2 leaky=2 ! \
    imxvideoconvert_g2d ! video/x-raw,width=300,height=300,format=RGBA ! videoconvert ! video/x-raw,format=RGB ! \      
    tensor_converter ! \
    tensor_transform mode=arithmetic option=typecast:float32,add:-127.5,div:127.5 ! \      
    tensor_filter framework=tensorflow-lite model=/home/root/nxp-nnstreamer-examples/detection/../downloads/models/detection/ssdlite_mobilenet_v2_coco_no_postprocess.tflite custom=Delegate:External,ExtDelegateLib:libvx_delegate.so ! \      
    tensor_decoder mode=bounding_boxes option1=mobilenet-ssd option2=/home/root/nxp-nnstreamer-examples/detection/../downloads/models/detection/coco_labels_list.txt option3=/home/root/nxp-nnstreamer-examples/detection/../downloads/models/detection/box_priors.txt option4=640:480 option5=300:300 ! \
    videoconvert ! \      
    mix. \    
    t. ! queue name=thread-img max-size-buffers=2 leaky=2 ! \      
    videoconvert ! \
    imxcompositor_g2d name=mix sink_0::zorder=2 sink_1::zorder=1 latency=300000000 min-upstream-latency=300000000 ! \      
    fpsdisplaysink video-sink=$VID_SINK name=test sync=false text-overlay=true -v 2>&1

错误日志 #

...
ERROR: Invoke called on model that is not ready.

** (gst-launch-1.0:1613): CRITICAL **: 21:30:40.018: Failed to invoke

** (gst-launch-1.0:1613): CRITICAL **: 21:30:40.019: /usr/lib/libnnstreamer-single.so(_backtrace_to_string+0x40) [0xffff6edb73c0]/usr/lib/gstreamer-1.0/libnnstreamer.so(+0x3d9ac) [0xffff6ee1d9ac]/usr/lib/libgstbase-1.0.so.0(+0x48704) [0xffff87198704]/usr/lib/libgstbase-1.0.so.0(+0x47dd4) [0xffff87197dd4]/usr/lib/libgstreamer-1.0.so.0(+0x8e908) [0xffff8783e908]/usr/lib/libgstreamer-1.0.so.0(+0x90798) [0xffff87840798]/usr/lib/libgstbase-1.0.so.0(+0x47ee8) [0xffff87197ee8]/usr/lib/libgstreamer-1.0.so.0(+0x8e908) [0xffff8783e908]/usr/lib/libgstreamer-1.0.so.0(+0x90798) [0xffff87840798]/usr/lib/gstreamer-1.0/libnnstreamer.so(+0x1fba0) [0xffff6edffba0]/usr/lib/libgstreamer-1.0.so.0(+0x8e908) [0xffff8783e908]/usr/lib/libgstreamer-1.0.so.0(+0x90798) [0xffff87840798]/usr/lib/libgstbase-1.0.so.0(+0x47ee8) [0xffff87197ee8]/usr/lib/libgstreamer-1.0.so.0(+0x8e908) [0xffff8783e908]/usr/lib/libgstreamer-1.0.so.0(+0x90798) [0xffff87840798]/usr/lib/libgstbase-1.0.so.0(+0x47ee8) [0xffff87197ee8]/usr/lib/libgstreamer-1.0.so.0(+0x8e908) [0xffff8783e908]/usr/lib/libgstreamer-1.0.so.0(+0x90798) [0xffff87840798]/usr/lib/libgstbase-1.0.so.0(+0x47ee8) [0xffff87197ee8]/usr/lib/libgstreamer-1.0.so.0(+0x8e908) [0xffff8783e908]


** (gst-launch-1.0:1613): CRITICAL **: 21:30:40.019: Calling invoke function (inference instance) of the tensor-filter subplugin (tensorflow-lite for /home/root/nxp-nnstreamer-examples/detection/../downloads/models/detection/ssdlite_mobilenet_v2_coco_no_postprocess.tflite) has failed with error code (-1).

ERROR: from element /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0: Internal data stream error.
Additional debug info:
../git/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0:
streaming stopped, reason error (-5)
Execution ended after 0:00:00.063685468
Setting pipeline to NULL ...
ERROR: from element /GstPipeline:pipeline0/GstQueue:thread-nn: Internal data stream error.
Additional debug info:
../git/plugins/elements/gstqueue.c(992): gst_queue_handle_sink_event (): /GstPipeline:pipeline0/GstQueue:thread-nn:
streaming stopped, reason error (-5)
...

系统信息 #

• Linux版本：Linux nitrogen8mp 5.15.71-2.2.0+yocto+gd4a57b30c1d1
• 通过自定义Yocto构建，已安装GStreamer、NNStreamer和TFLite VX Delegate

排查建议 #

1. 验证模型完整性与路径：确认ssdlite_mobilenet_v2_coco_no_postprocess.tflite模型文件路径正确，无损坏。用modelrunner直接加载该模型，验证是否能正常初始化。
2. 检查版本兼容性：确认Yocto构建时NNStreamer、TFLite、VX Delegate的版本匹配，尤其是NNStreamer的tensor-filter插件对TFLite delegate的支持版本是否兼容当前部署的库。
3. 调整流水线延迟参数：当前设置的latency=300000000可能不足以让模型完成初始化，尝试增大该值；或者移除min-upstream-latency参数，让GStreamer自动协商延迟。
4. 启用Delegate日志：在tensor_filter的custom参数中添加ExtDelegateOptions:Verbose:1，启用VX Delegate的详细日志，查看模型初始化阶段的具体错误，调整后的片段为：

   tensor_filter framework=tensorflow-lite model=/path/to/model.tflite custom=Delegate:External,ExtDelegateLib:libvx_delegate.so,ExtDelegateOptions:Verbose:1
   

5. 确认张量输入格式：检查tensor_transform处理后的张量维度、数据类型、归一化方式是否与模型要求完全匹配。可以用tensor_sink替换tensor_filter之后的组件，导出输入张量验证格式正确性。
6. 检查权限与库依赖：确保运行命令的用户有权限访问libvx_delegate.so和模型文件；用ldd libvx_delegate.so检查该库的依赖是否全部满足，无缺失系统库。

内容的提问来源于stack exchange，提问作者DragonflyRobotics