Android播放视频时，如何用FFmpeg/OpenGL在SurfaceView上绘制线条？

Hey there! Let's break down how to add that golf shot trajectory line over your Android video—whether you go the FFmpeg route or use OpenGL with SurfaceView. Both approaches work, but they fit different use cases, so I'll walk you through both.

If you don't need real-time interaction and just want to pre-process the video to include the trajectory line, FFmpeg is a solid choice. Here's how to do it:

• Define your trajectory parameters first: Figure out the pixel coordinates of the shot's start (x1, y1) and end (x2, y2) points on the video frame, plus the time window when the line should appear (e.g., 1 second after the shot until 3 seconds after).
• Use FFmpeg's drawline filter: This filter lets you draw a line directly onto video frames, with timing controls. Here's a sample command:

  ffmpeg -i input.mp4 -vf "drawline=x1=150:y1=300:x2=600:y2=700:color=yellow:thickness=4:enable='between(t,2,5)'" -c:a copy output.mp4
  

  Let's break this down:
  • x1/y1 = start coordinates, x2/y2 = end coordinates
  • color sets the line's hue (use hex codes like #FFFF00 if you want precise colors)
  • thickness controls how thick the line is
  • enable='between(t,2,5)' makes the line appear only between the 2nd and 5th second of the video
• Add animated trajectory (optional): If you want the line to draw gradually as the ball travels, use dynamic expressions for the coordinates. For example, to animate the line from start to end over 2 seconds:

  ffmpeg -i input.mp4 -vf "drawline=x1=150:y1=300:x2='150+(600-150)*((t-2)/2)':y2='300+(700-300)*((t-2)/2)':color=yellow:thickness=4:enable='between(t,2,4)'" -c:a copy output.mp4
  

• Integrate FFmpeg into Android: Use a pre-built library like MobileFFmpeg to run these commands directly in your app. Once processing is done, play the output video with a standard MediaPlayer or ExoPlayer.

If you need to overlay the trajectory while the video plays (e.g., for interactive adjustments or real-time events), combining OpenGL with GLSurfaceView (a subclass of SurfaceView optimized for OpenGL) is the way to go. Here's a step-by-step breakdown:

Core Setup #

1. Use GLSurfaceView: Replace your regular SurfaceView with GLSurfaceView, which handles the OpenGL rendering thread for you. Implement the GLSurfaceView.Renderer interface to control drawing.
2. Render the video as an OpenGL texture: Use MediaCodec to decode the video frames, then convert the YUV data into an OpenGL texture. Alternatively, use ExoPlayer's TextureView as a base, but GLSurfaceView gives you more control over overlay drawing.

Drawing the Trajectory Line #

In the renderer's onDrawFrame method, you'll first render the video texture, then draw the trajectory line on top:

• Convert pixel coordinates to OpenGL normalized coordinates: OpenGL uses a coordinate system where (-1,1) is the top-left corner and (1,-1) is the bottom-right. You'll need to map your video's pixel coordinates (e.g., 150,300 on a 1080p video) to this system.
• Create a simple shader program for lines: You'll need a vertex shader to define line positions and a fragment shader to set the line color.
• Draw the line in onDrawFrame: Here's a simplified code snippet for the drawing part:

  @Override
  public void onDrawFrame(GL10 gl) {
      // 1. Clear the screen
      GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT | GLES20.GL_DEPTH_BUFFER_BIT);
  
      // 2. Render the video texture (your existing video rendering code here)
      renderVideoTexture();
  
      // 3. Draw the trajectory line (only if we're in the correct time window)
      if (isTrajectoryVisible()) {
          GLES20.glUseProgram(lineShaderProgram);
  
          // Set line color (red in this example)
          int colorLoc = GLES20.glGetUniformLocation(lineShaderProgram, "vColor");
          float[] color = {1.0f, 0.0f, 0.0f, 1.0f};
          GLES20.glUniform4fv(colorLoc, 1, color, 0);
  
          // Set line vertices (normalized coordinates)
          int positionLoc = GLES20.glGetAttribLocation(lineShaderProgram, "vPosition");
          GLES20.glEnableVertexAttribArray(positionLoc);
          float[] lineVertices = {
              -0.6f, 0.4f, // Start point (mapped from pixel coords)
               0.5f, -0.3f  // End point
          };
          FloatBuffer vertexBuffer = ByteBuffer.allocateDirect(lineVertices.length * 4)
                  .order(ByteOrder.nativeOrder())
                  .asFloatBuffer()
                  .put(lineVertices);
          vertexBuffer.position(0);
          GLES20.glVertexAttribPointer(positionLoc, 2, GLES20.GL_FLOAT, false, 0, vertexBuffer);
  
          // Set line thickness
          GLES20.glLineWidth(3.0f);
  
          // Draw the line
          GLES20.glDrawArrays(GLES20.GL_LINES, 0, 2);
  
          GLES20.glDisableVertexAttribArray(positionLoc);
      }
  }
  

• Animate the line (optional): To make the line draw gradually, update the vertex coordinates each frame based on the video's current playback time. For example, calculate the percentage of the trajectory completed and adjust the end point accordingly.

Key Notes #

• Always handle coordinate conversions carefully—make sure your video's aspect ratio matches the GLSurfaceView's to avoid stretching the line.
• Use GLSurfaceView's setRenderMode(GLSurfaceView.RENDERMODE_CONTINUOUSLY) to ensure smooth animation of the trajectory line.

内容的提问来源于stack exchange，提问作者khushboo multani