Android端WebRTC如何录制视频流为MP4？基于AndroidRTC怎么保存为H.264格式？

Hey there! Let's walk through how to record WebRTC video streams on Android (using your AndroidRTC setup with libjingle_peerConnection and Socket.IO) and save them as H.264 first, then wrap that into an MP4 file. Here's a step-by-step breakdown:

WebRTC delivers video frames in YUV format, so we'll use Android's MediaCodec (hardware-accelerated for better performance) to encode these frames into H.264.

Step 1: Hook into the WebRTC Video Track #

In your AndroidRTC project, find where MediaStream objects are handled (usually in PeerConnectionClient or your main activity). When a stream is added (local or remote), grab the VideoTrack and attach a custom VideoSink to capture every frame:

// Example: Inside onAddStream callback for remote streams
@Override
public void onAddStream(MediaStream stream) {
    super.onAddStream(stream);
    if (!stream.videoTracks.isEmpty()) {
        VideoTrack videoTrack = stream.videoTracks.get(0);
        // Attach our custom sink to capture frames
        videoTrack.addSink(new VideoSink() {
            @Override
            public void onFrame(VideoFrame frame) {
                // Process each frame for encoding
                processVideoFrame(frame);
            }
        });
    }
}

Step 2: Initialize the H.264 Encoder #

Set up MediaCodec with parameters matching your stream's resolution, frame rate, and bitrate:

private MediaCodec h264Encoder;
private FileOutputStream h264OutputStream;

private void initH264Encoder(int width, int height, int fps, int bitrate) {
    try {
        // Configure H.264 encoding format
        MediaFormat format = MediaFormat.createVideoFormat(
            MediaFormat.MIMETYPE_VIDEO_AVC, width, height
        );
        format.setInteger(MediaFormat.KEY_COLOR_FORMAT, 
            MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420Flexible);
        format.setInteger(MediaFormat.KEY_BIT_RATE, bitrate);
        format.setInteger(MediaFormat.KEY_FRAME_RATE, fps);
        format.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 10); // Keyframe every 10 seconds

        // Create and start the encoder
        h264Encoder = MediaCodec.createEncoderByType(MediaFormat.MIMETYPE_VIDEO_AVC);
        h264Encoder.configure(format, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
        h264Encoder.start();

        // Initialize file output stream (use app private storage for Android 10+)
        File h264File = new File(getFilesDir(), "recording.h264");
        h264OutputStream = new FileOutputStream(h264File);
    } catch (IOException e) {
        e.printStackTrace();
    }
}

Step 3: Process & Encode Frames #

Convert WebRTC's VideoFrame into data the encoder can use, then write the encoded H.264 output to a file:

private void processVideoFrame(VideoFrame frame) {
    VideoFrame.I420Buffer i420Buffer = frame.getBuffer().toI420();

    // Get an available input buffer from the encoder
    int inputBufferIdx = h264Encoder.dequeueInputBuffer(-1);
    if (inputBufferIdx >= 0) {
        ByteBuffer inputBuffer = h264Encoder.getInputBuffers()[inputBufferIdx];
        inputBuffer.clear();

        // Write YUV data to the input buffer
        inputBuffer.put(i420Buffer.getDataY(), 0, 
            i420Buffer.getStrideY() * i420Buffer.getHeight());
        inputBuffer.put(i420Buffer.getDataU(), 0, 
            i420Buffer.getStrideU() * (i420Buffer.getHeight() / 2));
        inputBuffer.put(i420Buffer.getDataV(), 0, 
            i420Buffer.getStrideV() * (i420Buffer.getHeight() / 2));

        // Submit buffer to encoder (convert nanoseconds to microseconds)
        h264Encoder.queueInputBuffer(
            inputBufferIdx, 0, inputBuffer.position(), 
            frame.getTimestampNs() / 1000, 0
        );
    }

    // Read encoded output from the encoder
    MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();
    int outputBufferIdx = h264Encoder.dequeueOutputBuffer(bufferInfo, 0);
    while (outputBufferIdx >= 0) {
        ByteBuffer outputBuffer = h264Encoder.getOutputBuffers()[outputBufferIdx];
        byte[] h264Data = new byte[bufferInfo.size];
        outputBuffer.get(h264Data);

        // Write H.264 data to file
        if (h264OutputStream != null) {
            try {
                h264OutputStream.write(h264Data);
            } catch (IOException e) {
                e.printStackTrace();
            }
        }

        h264Encoder.releaseOutputBuffer(outputBufferIdx, false);
        outputBufferIdx = h264Encoder.dequeueOutputBuffer(bufferInfo, 0);
    }

    // Clean up the buffer
    i420Buffer.release();
}

Step 4: Stop Recording Properly #

Don't forget to clean up resources when you're done:

private void stopH264Recording() {
    if (h264Encoder != null) {
        h264Encoder.stop();
        h264Encoder.release();
        h264Encoder = null;
    }
    if (h264OutputStream != null) {
        try {
            h264OutputStream.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
        h264OutputStream = null;
    }
}

A raw H.264 file isn't playable on most devices—we need to package it into an MP4 container using MediaMuxer. Doing this while recording (instead of post-processing) is the most efficient approach.

Adjust the Setup for MP4 #

Modify the encoder initialization to include MediaMuxer:

private MediaMuxer mediaMuxer;
private int videoTrackIdx = -1;

private void initMP4Recording(String outputPath, int width, int height, int fps, int bitrate) {
    try {
        // Initialize muxer for MP4 output
        mediaMuxer = new MediaMuxer(outputPath, MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
        // Initialize encoder as before
        initH264Encoder(width, height, fps, bitrate);
    } catch (IOException e) {
        e.printStackTrace();
    }
}

Write Encoded Data to MP4 #

Update the output processing step to feed data into MediaMuxer instead of a raw file:

// Inside the output buffer loop
if ((bufferInfo.flags & MediaCodec.BUFFER_FLAG_CODEC_CONFIG) != 0) {
    // This is SPS/PPS data (required for MP4 container)
    MediaFormat outputFormat = h264Encoder.getOutputFormat();
    videoTrackIdx = mediaMuxer.addTrack(outputFormat);
    mediaMuxer.start();
} else if (videoTrackIdx != -1) {
    // Write encoded frame to MP4
    mediaMuxer.writeSampleData(videoTrackIdx, outputBuffer, bufferInfo);
}

Stop MP4 Recording #

Add muxer cleanup to the stop method:

private void stopMP4Recording() {
    stopH264Recording(); // Reuse the encoder cleanup logic
    if (mediaMuxer != null) {
        mediaMuxer.stop();
        mediaMuxer.release();
        mediaMuxer = null;
    }
}

• Get Stream Resolution: Grab the video width/height from the first VideoFrame you receive (use frame.getBuffer().getWidth() and getHeight()).
• Threading: Move all encoding/file operations to a background thread (use HandlerThread) to avoid blocking the UI.
• Permissions: For Android 10+, use app-specific storage (no external storage permission needed) or request MANAGE_EXTERNAL_STORAGE if you need to save to public directories. Add RECORD_AUDIO permission if you want to include audio in the MP4.

内容的提问来源于stack exchange，提问作者Mahdi Azadbar