求助：Android预览YUV_nv21帧转OpenCV Mat RGB后转回YUV_nv21用于录制

Hey there! I get how frustrating it can be hunting for this kind of specific conversion flow—especially when you already have the core logic working on Windows/Python. Let’s walk through exactly how to convert Android camera preview NV21 frames to OpenCV Mat (RGB), process them, and convert back to NV21 for recording.

Step 1: Convert NV21 Preview Frame to OpenCV RGB Mat #

Android camera preview frames come in NV21 format: a single byte array with a full Y plane first, followed by interleaved V/U values. Here’s how to turn that into an RGB Mat:

// Assume you get these from your camera preview callback:
byte[] nv21Data = ...;
int frameWidth = ...;
int frameHeight = ...;

// Create a Mat to hold the raw NV21 data (total pixels: width * height * 1.5)
Mat nv21Mat = new Mat(frameHeight + frameHeight/2, frameWidth, CvType.CV_8UC1);
nv21Mat.put(0, 0, nv21Data);

// Convert NV21 to RGB Mat using OpenCV's built-in color conversion
Mat rgbMat = new Mat();
Imgproc.cvtColor(nv21Mat, rgbMat, Imgproc.COLOR_YUV2RGB_NV21);

// Don't forget to release the NV21 Mat to avoid memory leaks
nv21Mat.release();

Quick Note on Frame Orientation #

Android camera previews are often rotated relative to the screen (e.g., 90 degrees for portrait mode). If your processed image looks flipped/rotated, fix it with OpenCV’s rotation function before processing:

// Rotate RGB Mat 90 degrees clockwise (adjust angle as needed)
Imgproc.rotate(rgbMat, rgbMat, Imgproc.ROTATE_90_CLOCKWISE);

Step 2: Run Your OpenCV Processing #

This part’s the easy one—use the same logic you built in your Python prototype! Whether it’s edge detection, filtering, or object detection, apply it directly to the rgbMat you just created.

Step 3: Convert Processed RGB Mat Back to NV21 #

Getting back to NV21 is a bit trickier, but there are two reliable ways depending on your OpenCV version:

Option 1: Use OpenCV’s Direct Conversion (Recommended for Newer Versions) #

If you’re using OpenCV 4.5+, the SDK includes a direct COLOR_RGB2YUV_NV21 flag:

Mat nv21OutputMat = new Mat();
Imgproc.cvtColor(rgbMat, nv21OutputMat, Imgproc.COLOR_RGB2YUV_NV21);

// Convert the Mat back to a byte array for recording
byte[] nv21OutputData = new byte[nv21OutputMat.rows() * nv21OutputMat.cols()];
nv21OutputMat.get(0, 0, nv21OutputData);

// Clean up
nv21OutputMat.release();
rgbMat.release();

Option 2: Manual I420 → NV21 Conversion (For Older OpenCV Versions) #

If the direct flag isn’t available, convert RGB to I420 first, then rearrange the U/V channels to match NV21’s interleaved VU format:

// Convert RGB to I420 YUV (separate Y, U, V planes)
Mat i420Mat = new Mat();
Imgproc.cvtColor(rgbMat, i420Mat, Imgproc.COLOR_RGB2YUV_I420);

// Extract individual Y, U, V plane data
byte[] yData = new byte[frameWidth * frameHeight];
byte[] uData = new byte[frameWidth * frameHeight / 4];
byte[] vData = new byte[frameWidth * frameHeight / 4];

i420Mat.rowRange(0, frameHeight).get(0, 0, yData);
i420Mat.rowRange(frameHeight, frameHeight + frameHeight/4).get(0, 0, uData);
i420Mat.rowRange(frameHeight + frameHeight/4, frameHeight + frameHeight/2).get(0, 0, vData);

// Combine into NV21: Y plane first, then interleaved V/U values
byte[] nv21OutputData = new byte[frameWidth * frameHeight * 3 / 2];
System.arraycopy(yData, 0, nv21OutputData, 0, yData.length);

int uvIndex = yData.length;
for (int i = 0; i < uData.length; i++) {
    nv21OutputData[uvIndex++] = vData[i];
    nv21OutputData[uvIndex++] = uData[i];
}

// Clean up all Mats
i420Mat.release();
rgbMat.release();

Key Tips to Avoid Headaches #

• Memory Management: Always call release() on Mat objects after you’re done with them—especially in preview callbacks, where frames come in fast and memory leaks add up quickly.
• OpenCV SDK Setup: Make sure you’ve correctly integrated the OpenCV Android SDK into your project (either via static library or OpenCV Manager).
• Test with Small Frames: If you run into crashes, start with lower-resolution preview frames to narrow down memory or performance issues.

Since you already have the core processing working in Python, this conversion layer should be the last piece to get your Android version up and running. Good luck!

内容的提问来源于stack exchange，提问作者methoneo