如何将GLSurfaceView渲染内容保存至文件？基于VidEffects库的技术求助

Hey there! Let's break down how to save your filtered video (using VideoSurfaceView from VidEffects) into an MP4 (H.264) file. The library focuses on real-time rendering, so we'll need to combine it with Android's media encoding APIs to capture and save the processed frames.

Core Approach #

The process boils down to three key steps:

1. Capture the filtered frames from your GLSurfaceView after the shader effects are applied.
2. Encode these frames into H.264 format using MediaCodec.
3. Package the encoded video (plus audio if needed) into an MP4 file with MediaMuxer.

Step 1: Capture Filtered Frames from GLSurfaceView #

Since VidEffects uses a custom GLVideoRenderer to apply shaders, you can extend this renderer to capture frames right after they're drawn. Override onDrawFrame to trigger frame capture once the shader has finished rendering:

public class CaptureEnabledRenderer extends GLVideoRenderer {
    private FrameCaptureCallback captureCallback;
    private int viewWidth, viewHeight;

    public CaptureEnabledRenderer(Context context, String videoPath, FrameCaptureCallback callback) {
        super(context, videoPath);
        this.captureCallback = callback;
    }

    @Override
    public void onSurfaceChanged(GL10 gl, int width, int height) {
        super.onSurfaceChanged(gl, width, height);
        this.viewWidth = width;
        this.viewHeight = height;
    }

    @Override
    public void onDrawFrame(GL10 gl) {
        super.onDrawFrame(gl);
        // Capture the frame after shader effects are applied
        if (captureCallback != null) {
            ByteBuffer rgbaFrame = ByteBuffer.allocateDirect(viewWidth * viewHeight * 4);
            GLES20.glReadPixels(0, 0, viewWidth, viewHeight, GLES20.GL_RGBA, GLES20.GL_UNSIGNED_BYTE, rgbaFrame);
            captureCallback.onFrameCaptured(rgbaFrame, viewWidth, viewHeight);
        }
    }

    public interface FrameCaptureCallback {
        void onFrameCaptured(ByteBuffer rgbaFrame, int width, int height);
    }
}

Step 2: Convert RGBA Frames to YUV for Encoding #

MediaCodec expects YUV-format data (typically NV21 or YUV420) instead of the RGBA data we get from glReadPixels. Use Android's RenderScript for efficient format conversion:

private void convertRgbaToYuv(ByteBuffer rgbaBuffer, int width, int height) {
    RenderScript rs = RenderScript.create(getContext());
    ScriptIntrinsicRGBToYuv rgbToYuv = ScriptIntrinsicRGBToYuv.create(rs, Element.U8(rs));

    // Convert ByteBuffer to Bitmap (for RenderScript input)
    Bitmap frameBitmap = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888);
    rgbaBuffer.rewind();
    frameBitmap.copyPixelsFromBuffer(rgbaBuffer);

    // Allocate buffers for conversion
    Allocation inAlloc = Allocation.createFromBitmap(rs, frameBitmap);
    int yuvBufferSize = width * height * 3 / 2; // YUV420 size
    Allocation outAlloc = Allocation.createSized(rs, Element.U8(rs), yuvBufferSize);

    // Run conversion
    rgbToYuv.setInput(inAlloc);
    rgbToYuv.forEach(outAlloc);

    // Extract YUV data
    ByteBuffer yuvBuffer = ByteBuffer.allocate(yuvBufferSize);
    outAlloc.copyTo(yuvBuffer);

    // Feed YUV data to encoder
    feedMediaCodec(yuvBuffer);
}

Step 3: Configure MediaCodec and MediaMuxer #

Set up the H.264 encoder and MP4 muxer to write the encoded frames to a file:

private MediaCodec mediaCodec;
private MediaMuxer mediaMuxer;
private int videoTrackIndex;
private long presentationTimeUs = 0;
private static final int FRAME_RATE = 30; // Match your source video's FPS

private void initEncoding(String outputPath, int width, int height) throws IOException {
    // Configure MediaFormat for H.264
    MediaFormat videoFormat = MediaFormat.createVideoFormat(
        MediaFormat.MIMETYPE_VIDEO_AVC, width, height
    );
    videoFormat.setInteger(MediaFormat.KEY_BIT_RATE, 2_000_000); // 2Mbps (adjust based on quality)
    videoFormat.setInteger(MediaFormat.KEY_FRAME_RATE, FRAME_RATE);
    videoFormat.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 5); // Keyframes every 5 seconds
    videoFormat.setInteger(MediaFormat.KEY_COLOR_FORMAT, 
        MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420Flexible
    );

    // Initialize encoder
    mediaCodec = MediaCodec.createEncoderByType(MediaFormat.MIMETYPE_VIDEO_AVC);
    mediaCodec.configure(videoFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
    mediaCodec.start();

    // Initialize muxer
    mediaMuxer = new MediaMuxer(outputPath, MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
}

private void feedMediaCodec(ByteBuffer yuvBuffer) {
    // Get input buffer from encoder
    int inputBufferIdx = mediaCodec.dequeueInputBuffer(1000);
    if (inputBufferIdx >= 0) {
        ByteBuffer inputBuffer = mediaCodec.getInputBuffer(inputBufferIdx);
        inputBuffer.clear();
        inputBuffer.put(yuvBuffer);

        // Calculate presentation time (keep video speed in sync)
        presentationTimeUs += 1_000_000 / FRAME_RATE;
        mediaCodec.queueInputBuffer(
            inputBufferIdx, 0, yuvBuffer.limit(), presentationTimeUs, 0
        );
    }

    // Process encoded output
    MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();
    int outputBufferIdx = mediaCodec.dequeueOutputBuffer(bufferInfo, 1000);
    while (outputBufferIdx >= 0) {
        ByteBuffer outputBuffer = mediaCodec.getOutputBuffer(outputBufferIdx);
        if ((bufferInfo.flags & MediaCodec.BUFFER_FLAG_CODEC_CONFIG) != 0) {
            // Skip codec config data
            bufferInfo.size = 0;
        }
        if (bufferInfo.size > 0) {
            // Add track to muxer if not already done
            if (videoTrackIndex == -1) {
                videoTrackIndex = mediaMuxer.addTrack(mediaCodec.getOutputFormat());
                mediaMuxer.start();
            }
            mediaMuxer.writeSampleData(videoTrackIndex, outputBuffer, bufferInfo);
        }
        mediaCodec.releaseOutputBuffer(outputBufferIdx, false);
        outputBufferIdx = mediaCodec.dequeueOutputBuffer(bufferInfo, 0);
    }
}

Step 4: Handle Audio (Optional) #

If you want to preserve the original video's audio, use MediaExtractor to extract the audio track, encode it (or reuse if format is compatible), and add it to the MediaMuxer alongside the video track. This ensures your saved MP4 has both video and audio.

Step 5: Clean Up Resources #

When you're done recording, make sure to release all resources properly:

private void stopEncoding() {
    // Signal encoder to finish
    mediaCodec.signalEndOfInputStream();

    // Drain remaining output
    MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();
    int outputBufferIdx;
    do {
        outputBufferIdx = mediaCodec.dequeueOutputBuffer(bufferInfo, 1000);
        if (outputBufferIdx >= 0) {
            if (bufferInfo.size > 0) {
                mediaMuxer.writeSampleData(videoTrackIndex, mediaCodec.getOutputBuffer(outputBufferIdx), bufferInfo);
            }
            mediaCodec.releaseOutputBuffer(outputBufferIdx, false);
        }
    } while (outputBufferIdx != MediaCodec.INFO_TRY_AGAIN_LATER);

    // Release resources
    mediaCodec.stop();
    mediaCodec.release();
    mediaMuxer.stop();
    mediaMuxer.release();
}

Key Notes #

• Performance: glReadPixels can be slow. For better performance, consider using EGL image/texture sharing to avoid copying pixel data between GPU and CPU.
• Frame Sync: Match the capture rate to your source video's FPS to avoid speed discrepancies in the saved file.
• Error Handling: Add try-catch blocks and error checking for all media API calls to handle edge cases (e.g., invalid formats, file write failures).

内容的提问来源于stack exchange，提问作者Nitin Verma