如何配置GStreamer用OpenGLES替代OpenGL适配瑞芯微Mali GPU？

Hey there, let's break down how to solve your problem with GStreamer and your Mali GPU that only supports OpenGLES:

1. Try Configuring the Existing gl Plugin for OpenGLES #

Good news: GStreamer's gl plugin can work with OpenGLES if you set up your Buildroot environment correctly. Here's what to check:

• In Buildroot's configuration menu (make menuconfig), navigate to Package Selection for the target > Multimedia > GStreamer 1.x
  • Make sure you enable gstreamer1-gl and specifically check the EGL/OpenGLES-related options like BR2_PACKAGE_GSTREAMER1_GL_EGL (this tells GStreamer to use EGL instead of GLX for context management)
  • Ensure your Mali GPU drivers (from Rockchip's BSP) are selected in Buildroot—this provides the necessary EGL and GLES libraries that GStreamer will link against
• After building, verify with gst-inspect-1.0 glrotate (or any gl filter) that the element lists EGL-based caps (look for something like video/x-raw(memory:GLMemory), format=RGBA, texture-target=2D and references to EGL)
• If configured correctly, you should be able to use the stock glrotate element with your GPU, e.g.:

  gst-launch-1.0 videotestsrc ! glrotate angle=52 ! autovideosink
  

  This should leverage your Mali GPU instead of falling back to slow software rendering.

2. Build a Custom OpenGLES-Based Transform Plugin #

If for some reason the stock gl plugin doesn't cooperate, writing a lightweight custom plugin is totally feasible. Here's a quick guide and structure:

Core Concept #

We'll create a GstVideoFilter element that uses EGL for context management and OpenGLES 2.0 for applying rotation transforms. The plugin will:

• Initialize an EGL context tied to your display
• Compile simple GLES shaders to handle vertex transformation
• Upload input video frames to GLES textures, apply the rotation matrix, and render back to an output frame

Basic Code Structure #

Header (myplugin.h)

#ifndef MYPLUGIN_H
#define MYPLUGIN_H

#include <gst/gst.h>
#include <gst/video/gstvideofilter.h>
#include <EGL/egl.h>
#include <GLES2/gl2.h>

G_BEGIN_DECLS

#define GST_TYPE_MYPLUGIN (gst_myplugin_get_type())
G_DECLARE_FINAL_TYPE(GstMyPlugin, gst_myplugin, GST, MYPLUGIN, GstVideoFilter)

struct _GstMyPlugin {
  GstVideoFilter parent;

  // Rotation angle property
  gfloat angle;

  // EGL/GLES resources
  EGLDisplay egl_display;
  EGLContext egl_context;
  GLuint program;
  GLint angle_loc;
};

G_END_DECLS

#endif // MYPLUGIN_H

Source (myplugin.c)

You'll need to implement standard GStreamer plugin boilerplate, plus:

• gst_myplugin_set_caps: Initialize EGL context and GLES shaders here
• gst_myplugin_filter_frame: Handle frame upload, apply rotation via shader uniforms, render to output frame
• Property handling for x-rot (map it to the angle field in the struct)

Buildroot Integration #

Add your plugin as an external package in Buildroot:

1. Create a myplugin directory with your source files and a myplugin.mk file:

   MYPLUGIN_VERSION = 0.1
   MYPLUGIN_SITE = $(TOPDIR)/../path/to/myplugin
   MYPLUGIN_SITE_METHOD = local
   MYPLUGIN_DEPENDENCIES = gstreamer1 gstreamer1-video libgles2 libegl
   MYPLUGIN_INSTALL_STAGING = YES
   
   define MYPLUGIN_BUILD_CMDS
       $(MAKE) CC=$(TARGET_CC) CFLAGS="$(TARGET_CFLAGS) -I$(STAGING_DIR)/usr/include" LDFLAGS="$(TARGET_LDFLAGS) -L$(STAGING_DIR)/usr/lib -lgstreamer-1.0 -lgstvideo-1.0 -lGLESv2 -lEGL" -C $(@D)
   endef
   
   define MYPLUGIN_INSTALL_TARGET_CMDS
       $(INSTALL) -D $(@D)/libmyplugin.so $(TARGET_DIR)/usr/lib/gstreamer-1.0/libmyplugin.so
   endef
   
   $(eval $(generic-package))
   

2. Add the package to your Buildroot .config: BR2_PACKAGE_MYPLUGIN=y

Test the Custom Plugin #

Once built, you can run your desired pipeline:

gst-launch-1.0 videotestsrc ! myplugin x-rot=52 ! autovideosink

Final Notes #

• Always double-check that your Mali drivers are correctly installed and that EGL is properly configured (use eglinfo to verify EGL capabilities)
• If you need more complex transforms beyond rotation, you can extend the shader logic or add more properties to your custom plugin

内容的提问来源于stack exchange，提问作者DMacAttack