MediaFormat的KEY_ROTATION选项在谷歌参考机型失效求助

Problem Description #

I'm facing an issue with video rotation during MediaCodec encoding: setting MediaFormat.KEY_ROTATION works perfectly on Samsung Galaxy S6 (Nougat), S7 (Nougat), and S8 (Nougat) devices—these models correctly rotate the pixel data in the output buffer. However, on Google reference devices like the Pixel 2 XL (Oreo) and Nexus 5 (Lollipop), this setting has no effect at all; the output buffer content remains unrotated.

My setup uses a Surface created via encoderSurface() as the input source, and I handle output buffers through MediaCodec.Callback's onOutputBufferAvailable method. Below is the relevant code:

Relevant Code #

MediaCodec Callback Implementation

private final MediaCodec.Callback _mediaCodecCallback = new MediaCodec.Callback() { 
    @Override public void finalize(){ } 

    @Override public void onInputBufferAvailable(@NonNull MediaCodec mediaCodec, int i) { } 

    @Override public void onOutputBufferAvailable(@NonNull MediaCodec mediaCodec, int i, @NonNull MediaCodec.BufferInfo bufferInfo) { 
        ByteBuffer buffer = mediaCodec.getOutputBuffer(i); 
        byte[] outData = new byte[bufferInfo.size]; 
        buffer.get(outData); 
        mediaCodec.releaseOutputBuffer(i, false); 

        switch (bufferInfo.flags) { 
            case MediaCodec.BUFFER_FLAG_CODEC_CONFIG: 
                if(DEBUG_LOG) { 
                    Log.v(TAG, "CONFIG FRAME"); 
                } 
                _configFrame = new byte[bufferInfo.size]; 
                System.arraycopy(outData, 0, _configFrame, 0, outData.length); 
                break; 
            case MediaCodec.BUFFER_FLAG_KEY_FRAME: 
                // I Frame; 
                if(DEBUG_LOG) { 
                    Log.v(TAG, "I FRAME" + ", SIZE = " + bufferInfo.size + ", " + currentDateandTime); 
                } 
                try { 
                    _outputFrame.reset(); 
                    _outputFrame.write(_configFrame); 
                    _outputFrame.write(outData); 
                } catch (IOException e) { 
                    e.printStackTrace(); 
                } 
                break; 
            default: 
                // P Frame; 
                if(DEBUG_LOG) { 
                    Log.v(TAG, "P FRAME" + ", SIZE = " + bufferInfo.size); 
                } 
                break; 
        } 
    } 
};

Encoder Initialization Code

public boolean initialize(int width,int height) { 
    try { 
        _mediaCodec = MediaCodec.createEncoderByType(MIME_TYPE); 
    } catch (IOException e) { 
        e.printStackTrace(); 
        return false; 
    } 

    MediaCodecInfo codec = selectCodec(MIME_TYPE); 
    if (codec == null) { 
        return false; 
    } 

    MediaCodecInfo.CodecCapabilities cap = codec.getCapabilitiesForType(MIME_TYPE); 
    MediaFormat mediaFormat = MediaFormat.createVideoFormat(MIME_TYPE, width, height); 
    mediaFormat.setInteger(MediaFormat.KEY_FRAME_RATE, 30); 
    mediaFormat.setInteger(MediaFormat.KEY_COLOR_FORMAT, MediaCodecInfo.CodecCapabilities.COLOR_FormatSurface); 
    mediaFormat.setInteger(MediaFormat.KEY_BIT_RATE, bitRateInfo.getInstance().getBitRate()); // 300000 
    mediaFormat.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 1); 
    mediaFormat.setInteger(MediaFormat.KEY_ROTATION, 90); 
    // in under API 21, not use MediaFormat.KEY_ROTATION, but use "rotation-degrees" 
    // This does not work on a Google reference phone. 

    _mediaCodec.configure(mediaFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE); 
    _surface = _mediaCodec.createInputSurface(); 
    _mediaCodec.setCallback(_mediaCodecCallback); 
    _mediaCodec.start(); 
    Log.d(TAG,"Encoder Start SUCCESS"); 
    return true; 
} 

public Surface encoderSurface() { 
    //Return the value of this function and use it as the input surface. 
    return _surface; 
}

Root Cause Analysis #

The core issue here lies in vendor-specific implementations of MediaCodec:

• Samsung's custom implementation for Nougat devices interprets KEY_ROTATION as a command to physically rotate the pixel data in the output buffer when using a Surface input.
• Google's native MediaCodec implementation (used in Pixel/Nexus devices) treats KEY_ROTATION as a metadata setting only. It adds rotation information to the video stream's container/headers but does not modify the raw pixel data in the output buffer. Since you're directly reading the buffer's byte content, you won't see any visual rotation on these devices.

Additionally, note that for API levels below 21, KEY_ROTATION isn't officially supported (you need to use the string key "rotation-degrees"), but even on API 21+ Google devices, the behavior remains metadata-only for Surface inputs.

Solutions #

1. Pre-Rotate Frames Before Rendering to Encoder Surface (Recommended)

The most reliable cross-device solution is to manually rotate your input frames before rendering them to the encoder's Surface. This ensures consistent behavior across all devices, regardless of vendor implementation.

You can achieve this using OpenGL ES:

• Create an EGL context tied to the encoder's Surface.
• Use a shader to rotate the input texture by 90 degrees (or your desired rotation) before drawing it to the encoder's Surface.

This approach avoids relying on vendor-specific codec behavior and guarantees the output buffer contains pre-rotated pixel data.

2. Handle Rotation During Playback (If Applicable)

If your goal is just to have the video play back correctly (rather than modifying the raw pixel data in the output buffer), you can read the rotation metadata from the encoded stream and apply the rotation during playback. For example, using MediaMetadataRetriever to get the rotation value and setting it on your VideoView or ExoPlayer instance.

However, this won't change the raw pixel data in the output buffer, so it's only useful if you don't need the rotated pixels directly.

3. Vendor-Specific Adaptation (Not Recommended)

You could detect if the device is a Google reference model and apply manual rotation only for those cases, while relying on KEY_ROTATION for Samsung devices. This approach is fragile, though, as vendor behaviors can change with OS updates, and it increases maintenance overhead.

Final Notes #

Always test video encoding behavior across multiple devices and OS versions, since MediaCodec implementations can vary significantly between vendors. Pre-rotating frames via OpenGL ES is the most robust way to ensure consistent rotation handling.

内容的提问来源于stack exchange，提问作者SKWyverns29