基于byte[]的实时图像处理APP技术咨询：NV21与JPEG处理方案

Hey there! Let's break down your questions clearly—since you're focusing on direct byte[] operations for real-time performance, that's the right approach, and we can avoid unnecessary format conversions that slow things down.

First: Why Processing JPEG Byte[] Isn't the Way to Go #

You noticed that converting NV21 to JPEG didn't give you 1 or 3 bytes per pixel—this is because JPEG is a compressed format, not raw pixel data. The bytes in a JPEG file are encoded (not direct pixel values), so you can't do per-pixel tasks like binarization or color inversion directly on it. You'd have to decode it first, which kills your performance goal.

So let's focus on working with the raw NV21 data from your camera preview, or converting it to RGB byte[] efficiently.

Question 2: Convert NV21 Byte[] to RGB Byte[] #

NV21 is a YUV420sp format, meaning:

• The first width * height bytes are the Y (luminance) channel (1 byte per pixel)
• The remaining width * height / 2 bytes are UV (chrominance) channels, stored as alternating V and U values (4 Y pixels share one UV pair)

Here's a straightforward Java implementation to convert NV21 to RGB byte[] (each pixel takes 3 bytes: R, G, B in order):

public static byte[] nv21ToRgb(byte[] nv21, int width, int height) {
    int totalPixels = width * height;
    byte[] rgbOutput = new byte[totalPixels * 3];
    
    int y, u, v;
    for (int row = 0; row < height; row++) {
        int uvStartIndex = totalPixels + (row / 2) * width;
        int rgbRowStart = row * width * 3;
        
        for (int col = 0; col < width; col++) {
            // Extract Y, U, V values
            y = nv21[row * width + col] & 0xFF;
            u = nv21[uvStartIndex + (col & ~1)] & 0xFF; // Even index for U
            v = nv21[uvStartIndex + (col & 1)] & 0xFF;  // Odd index for V
            
            // YUV to RGB conversion formula (standard BT.601)
            int r = (int) (y + 1.370705 * (v - 128));
            int g = (int) (y - 0.698001 * (v - 128) - 0.337633 * (u - 128));
            int b = (int) (y + 1.732446 * (u - 128));
            
            // Clamp values to 0-255 to avoid overflow
            r = Math.max(0, Math.min(r, 255));
            g = Math.max(0, Math.min(g, 255));
            b = Math.max(0, Math.min(b, 255));
            
            // Assign to RGB array
            rgbOutput[rgbRowStart + col * 3] = (byte) r;
            rgbOutput[rgbRowStart + col * 3 + 1] = (byte) g;
            rgbOutput[rgbRowStart + col * 3 + 2] = (byte) b;
        }
    }
    return rgbOutput;
}

Note: For higher resolution frames, consider moving this conversion to JNI/NDK (C/C++)—it'll run much faster than Java loops.

Question 1: Binarization & Color Inversion on Raw Byte[] #

Let's cover both operations, either directly on NV21 (for better performance) or on RGB.

1. Binarization #

Binarization turns the image into black and white based on a brightness threshold. It's most efficient to do this directly on the Y channel of NV21:

Binarize NV21 Directly

public static void binarizeNv21(byte[] nv21, int width, int height, int threshold) {
    int yChannelSize = width * height;
    
    // Process Y channel (set to white/black based on threshold)
    for (int i = 0; i < yChannelSize; i++) {
        int yValue = nv21[i] & 0xFF;
        nv21[i] = (byte) (yValue >= threshold ? 0xFF : 0x00);
    }
    
    // Optional: Set UV channels to neutral (128) to avoid color artifacts
    for (int i = yChannelSize; i < nv21.length; i++) {
        nv21[i] = (byte) 0x80;
    }
}

Binarize RGB Byte[]

If you're working with RGB data, convert to grayscale first then apply the threshold:

public static void binarizeRgb(byte[] rgb, int width, int height, int threshold) {
    int totalPixels = width * height;
    
    for (int i = 0; i < totalPixels; i++) {
        int r = rgb[i*3] & 0xFF;
        int g = rgb[i*3+1] & 0xFF;
        int b = rgb[i*3+2] & 0xFF;
        
        // Calculate grayscale value (standard luminance formula)
        int gray = (int) (0.299 * r + 0.587 * g + 0.114 * b);
        
        // Apply threshold
        byte binaryVal = (byte) (gray >= threshold ? 0xFF : 0x00);
        rgb[i*3] = binaryVal;
        rgb[i*3+1] = binaryVal;
        rgb[i*3+2] = binaryVal;
    }
}

2. Color Inversion #

Inversion flips each pixel's brightness/color to its opposite. Again, you can do this on NV21 or RGB.

Invert NV21 Directly

public static void invertNv21(byte[] nv21) {
    int yChannelSize = nv21.length * 2 / 3; // Y is 2/3 of NV21's total length
    
    // Invert Y channel (grayscale negative)
    for (int i = 0; i < yChannelSize; i++) {
        nv21[i] = (byte) (0xFF - (nv21[i] & 0xFF));
    }
    
    // Optional: Invert UV channels for full color inversion
    for (int i = yChannelSize; i < nv21.length; i++) {
        nv21[i] = (byte) (0xFF - (nv21[i] & 0xFF));
    }
}

Invert RGB Byte[]

This is simple—just flip each R, G, B value:

public static void invertRgb(byte[] rgb) {
    for (int i = 0; i < rgb.length; i++) {
        rgb[i] = (byte) (0xFF - (rgb[i] & 0xFF));
    }
}

Final Tips for Real-Time Performance #

• Skip JPEG conversion entirely—it adds unnecessary compression/decompression overhead that kills real-time speed.
• Use NV21 directly for operations like binarization (you only need the Y channel) to save memory and processing time.
• For full color operations, convert NV21 to RGB once, then process the RGB byte[].
• For large resolution frames, use JNI/NDK for pixel-level loops—C/C++ is significantly faster than Java for these tasks.

内容的提问来源于stack exchange，提问作者Unknownpgr