JNI结合OpenCV处理CameraPreview：实时绘制矩形异常求助

Hey there! Let's break down what's going wrong with your CameraPreview processing and fix those issues step by step. Since you're new to C/C++ and JNI, I'll keep explanations clear and provide actionable code snippets you can adapt directly.

Camera preview frames are almost always in YUV format (most commonly NV21), while Android's Bitmap uses ARGB_8888 by default. If you skip proper conversion between these formats, you'll end up with a gray image—because you're only displaying the brightness (Y) channel and ignoring color (UV) data.

Common Mistakes & Fixes #

• Missing YUV-to-ARGB conversion: You need to translate the YUV pixel data into the ARGB format the Bitmap expects.
• Incorrect pixel indexing: Messing up how you map Y/UV values to ARGB positions will break color rendering.

JNI Code for Proper NV21-to-ARGB Conversion

This is a ready-to-use JNI function that handles the conversion correctly. Add it to your native code file:

#include <jni.h>
#include <android/bitmap.h>
#include <android/log.h>

// Helper for debug logs (optional but useful)
#define LOG_TAG "CameraJNI"
#define LOGD(...) __android_log_print(ANDROID_LOG_DEBUG, LOG_TAG, __VA_ARGS__)

// Core conversion: NV21 (Camera frame) → ARGB_8888 (Bitmap)
void nv21ToArgb(const unsigned char* nv21, int width, int height, unsigned char* argb) {
    int frameSize = width * height;
    int yIndex, uvIndex;
    unsigned char y, u, v;
    int r, g, b;

    for (int j = 0; j < height; j++) {
        for (int i = 0; i < width; i++) {
            // Calculate positions for Y (brightness) and UV (color) channels
            yIndex = j * width + i;
            uvIndex = frameSize + (j / 2) * width + (i - i % 2);

            y = nv21[yIndex];
            u = nv21[uvIndex];
            v = nv21[uvIndex + 1];

            // Standard YUV-to-RGB conversion formula (BT.601)
            r = (int)(y + 1.370705 * (v - 128));
            g = (int)(y - 0.698001 * (v - 128) - 0.337633 * (u - 128));
            b = (int)(y + 1.732446 * (u - 128));

            // Clamp values to valid 0-255 range
            r = r < 0 ? 0 : (r > 255 ? 255 : r);
            g = g < 0 ? 0 : (g > 255 ? 255 : g);
            b = b < 0 ? 0 : (b > 255 ? 255 : b);

            // Write to ARGB_8888 buffer: Alpha (255 = opaque), Red, Green, Blue
            argb[yIndex * 4 + 0] = 255;
            argb[yIndex * 4 + 1] = (unsigned char)r;
            argb[yIndex * 4 + 2] = (unsigned char)g;
            argb[yIndex * 4 + 3] = (unsigned char)b;
        }
    }
}

// JNI method called from Java to process the frame
extern "C" JNIEXPORT void JNICALL
Java_com_your_package_YourClassName_processCameraFrame(
        JNIEnv* env, jobject thiz,
        jbyteArray nv21_frame, jint width, jint height, jobject bitmap) {

    AndroidBitmapInfo bitmap_info;
    void* bitmap_pixels;

    // Step 1: Validate Bitmap format and lock its pixel buffer
    if (AndroidBitmap_getInfo(env, bitmap, &bitmap_info) != ANDROID_BITMAP_RESULT_SUCCESS) {
        LOGD("Failed to get Bitmap info");
        return;
    }
    if (bitmap_info.format != ANDROID_BITMAP_FORMAT_ARGB_8888) {
        LOGD("Bitmap must be ARGB_8888 format");
        return;
    }
    if (AndroidBitmap_lockPixels(env, bitmap, &bitmap_pixels) != ANDROID_BITMAP_RESULT_SUCCESS) {
        LOGD("Failed to lock Bitmap pixels");
        return;
    }

    // Step 2: Get pointer to the input NV21 frame data
    jbyte* nv21_ptr = env->GetByteArrayElements(nv21_frame, nullptr);
    if (nv21_ptr == nullptr) {
        AndroidBitmap_unlockPixels(env, bitmap);
        return;
    }

    // Step 3: Run conversion and write to Bitmap
    nv21ToArgb((unsigned char*)nv21_ptr, width, height, (unsigned char*)bitmap_pixels);

    // Step 4: Clean up resources
    env->ReleaseByteArrayElements(nv21_frame, nv21_ptr, 0);
    AndroidBitmap_unlockPixels(env, bitmap);
}

Java Side Setup

Make sure you create a Bitmap with the correct format and pass it to the JNI method:

// Create a mutable ARGB_8888 Bitmap matching the preview size
Bitmap processedBitmap = Bitmap.createBitmap(previewWidth, previewHeight, Bitmap.Config.ARGB_8888);

// Call JNI to process the NV21 frame
yourJNIInstance.processCameraFrame(nv21FrameData, previewWidth, previewHeight, processedBitmap);

// Update the ImageView with the processed Bitmap
imageView.setImageBitmap(processedBitmap);

This happens when your ImageView isn't properly covering the CameraPreview, or you're drawing on the wrong layer. Here's how to fix it:

Common Causes & Solutions #

1. ImageView is transparent: Ensure it's fully opaque so the CameraPreview doesn't show through.

   imageView.setAlpha(1.0f);
   imageView.setBackgroundColor(Color.BLACK); // Cover any gaps
   

2. Drawing on the wrong layer: Always draw on the processed Bitmap first, then set it to the ImageView—don't draw directly on the CameraPreview.

   // Make a mutable copy of the processed Bitmap (required for drawing)
   Bitmap editableBitmap = processedBitmap.copy(Bitmap.Config.ARGB_8888, true);
   Canvas canvas = new Canvas(editableBitmap);
   
   // Draw your rectangle (example: red fill)
   Paint paint = new Paint();
   paint.setColor(Color.RED);
   paint.setStyle(Paint.Style.FILL);
   canvas.drawRect(100, 100, 300, 300, paint);
   
   // Update the ImageView with the edited Bitmap
   imageView.setImageBitmap(editableBitmap);
   

3. Layout hierarchy issue: Ensure the ImageView is positioned on top of the CameraPreview in your layout. In XML, place the ImageView after the CameraPreview:

   <FrameLayout
       android:layout_width="match_parent"
       android:layout_height="match_parent">
   
       <!-- Camera preview (bottom layer) -->
       <com.your.package.CameraPreview
           android:layout_width="match_parent"
           android:layout_height="match_parent"/>
   
       <!-- Processed image (top layer) -->
       <ImageView
           android:id="@+id/processedImageView"
           android:layout_width="match_parent"
           android:layout_height="match_parent"
           android:scaleType="fitCenter"/>
   </FrameLayout>
   

• Test without JNI first: Convert the NV21 frame to a Bitmap in Java using YuvImage to confirm the frame data itself is valid. If that works, the problem is in your JNI code.
• Add debug logs: Use the LOGD macros in the JNI code to check if width/height values match, and if the Bitmap format is correct.

内容的提问来源于stack exchange，提问作者Tatsuo