如何修复Android端Google ML+CameraX目标检测boundingBox坐标偏移问题

API 探路者

2026-10-7

Android端Google ML+CameraX目标检测边界框偏移问题

我正在使用Java开发Android端目标检测应用，采用Google ML和CameraX框架，使用的TensorFlow模型为公开的mobilenet_v1_0.75_192_quantized量化模型。目前遇到的问题是检测输出的boundingBox坐标存在轻微偏移，如下图所示，请忽略检测结果将物品识别为刮刀的问题，我当前的核心诉求是让boundingBox准确框住屏幕显示画面中的对应物体。
边界框偏移示例
以下是用于绘制graphicOverlay的相关代码：

DrawGraphic.java

public class DrawGraphic extends View {

    Paint borderPaint, textPaint;
    Rect rect;
    String text;

    ImageProxy imageProxy;
    PreviewView previewView;


    public DrawGraphic(Context context, Rect rect, String text, ImageProxy imageProxy, PreviewView previewView) {
        super(context);
        this.rect = rect;
        this.text = text;
        this.imageProxy = imageProxy;
        this.previewView = previewView;

        borderPaint = new Paint();
        borderPaint.setColor(Color.WHITE);
        borderPaint.setStrokeWidth(10f);
        borderPaint.setStyle(Paint.Style.STROKE);

        textPaint = new Paint();
        textPaint.setColor(Color.WHITE);
        textPaint.setStrokeWidth(50f);
        textPaint.setTextSize(32f);
        textPaint.setStyle(Paint.Style.FILL);
    }

    @Override
    protected void onDraw(Canvas canvas) {
        super.onDraw(canvas);
        canvas.setMatrix(getMappingMatrix(imageProxy, previewView));
        canvas.concat(getMappingMatrix(imageProxy, previewView));
        canvas.drawText(text, rect.centerX(), rect.centerY(), textPaint);
        canvas.drawRect(rect.left, rect.bottom, rect.right, rect.top, borderPaint);
    }

    Matrix getMappingMatrix(ImageProxy imageProxy, PreviewView previewView) {
        Rect cropRect = imageProxy.getCropRect();
        int rotationDegrees = imageProxy.getImageInfo().getRotationDegrees();
        Matrix matrix = new Matrix();

        float[] source = {
                cropRect.left,
                cropRect.top,
                cropRect.right,
                cropRect.top,
                cropRect.right,
                cropRect.bottom,
                cropRect.left,
                cropRect.bottom
        };

        float[] destination = {
                0f,
                0f,
                previewView.getWidth(),
                0f,
                previewView.getWidth(),
                previewView.getHeight(),
                0f,
                previewView.getHeight()
        };

        int vertexSize = 2;

        int shiftOffset = rotationDegrees / 90 * vertexSize;
        float[] tempArray = destination.clone();
        for (int toIndex = 0; toIndex < source.length; toIndex++) {
            int fromIndex = (toIndex + shiftOffset) % source.length;
            destination[toIndex] = tempArray[fromIndex];
        }
        matrix.setPolyToPoly(source, 0, destination, 0, 4);
        return matrix;
    }
}

MainActivity.java

public class MainActivity extends AppCompatActivity {

    private static final int PERMISSIONS_REQUEST = 1;

    private static final String PERMISSION_CAMERA = Manifest.permission.CAMERA;

    public static final Size DESIRED_PREVIEW_SIZE = new Size(640, 480);

    private PreviewView previewView;

    ActivityMainBinding binding;

    @Override
    protected void onCreate(@Nullable Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        binding = ActivityMainBinding.inflate(getLayoutInflater());
        setContentView(binding.getRoot());

        previewView = findViewById(R.id.previewView);

        if (hasPermission()) {
            // 启动CameraX
            startCamera();
        } else {
            requestPermission();
        }
    }

    @SuppressLint("UnsafeOptInUsageError")
    private void startCamera() {
        ListenableFuture<ProcessCameraProvider> cameraProviderFuture = ProcessCameraProvider.getInstance(this);

        cameraProviderFuture.addListener(() -> {
            try {
                ProcessCameraProvider cameraProvider = cameraProviderFuture.get();

                Preview preview = new Preview.Builder().build();

                CameraSelector cameraSelector = new CameraSelector.Builder()
                        .requireLensFacing(CameraSelector.LENS_FACING_BACK)
                        .build();

                ImageAnalysis imageAnalysis =
                        new ImageAnalysis.Builder()
                                .setTargetResolution(DESIRED_PREVIEW_SIZE)
                                .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
                                .build();

                imageAnalysis.setAnalyzer(ContextCompat.getMainExecutor(this), imageProxy -> {
                    int rotationDegrees = imageProxy.getImageInfo().getRotationDegrees();
                    Log.v("ImageAnalysis_degrees", String.valueOf(rotationDegrees));

                    @SuppressLint("UnsafeExperimentalUsageError") Image mediaImage = imageProxy.getImage();
                    if (mediaImage != null) {
                        InputImage image =
                                InputImage.fromMediaImage(mediaImage, imageProxy.getImageInfo().getRotationDegrees());

                        LocalModel localModel =
                                new LocalModel.Builder()
                                        .setAssetFilePath("mobilenet_v1_0.75_192_quantized_1_metadata_1.tflite")
                                        .build();

                        CustomObjectDetectorOptions customObjectDetectorOptions =
                                new CustomObjectDetectorOptions.Builder(localModel)
                                        .setDetectorMode(CustomObjectDetectorOptions.STREAM_MODE)
                                        .enableClassification()
                                        .setClassificationConfidenceThreshold(0.5f)
                                        .setMaxPerObjectLabelCount(3)
                                        .build();

                        ObjectDetector objectDetector =
                                ObjectDetection.getClient(customObjectDetectorOptions);

                        objectDetector.process(image)
                                .addOnSuccessListener(detectedObjects -> {
                                    getObjectResults(detectedObjects);
                                    Log.d("TAG", "onSuccess" + detectedObjects.size());
                                    for (DetectedObject detectedObject : detectedObjects) {
                                        Rect boundingBox = detectedObject.getBoundingBox();

                                        Integer trackingId = detectedObject.getTrackingId();
                                        for (DetectedObject.Label label : detectedObject.getLabels()) {
                                            String text = label.getText();
                                            int index = label.getIndex();
                                            float confidence = label.getConfidence();
                                        }
                                    }
                                })
                                .addOnFailureListener(e -> Log.e("TAG", e.getLocalizedMessage()))
                                .addOnCompleteListener(result -> imageProxy.close());
                    }

                });

                preview.setSurfaceProvider(
                        previewView.getSurfaceProvider());

                if (cameraProvider != null) {
                    Camera camera = cameraProvider.bindToLifecycle(
                            this,
                            cameraSelector,
                            imageAnalysis,
                            preview);
                }

            } catch (ExecutionException | InterruptedException e) {
                e.printStackTrace();
            }


        }, ContextCompat.getMainExecutor(this));
    }

    private void getObjectResults(List<DetectedObject> detectedObjects) {
        for (DetectedObject object : detectedObjects) {
            if (binding.parentlayout.getChildCount() > 1) {
                binding.parentlayout.removeViewAt(1);
            }
            Rect rect = object.getBoundingBox();
            String text = "Undefined";
            if (object.getLabels().size() != 0) {
                text = object.getLabels().get(0).getText();
            }

            DrawGraphic drawGraphic = new DrawGraphic(this, rect, text);
            binding.parentlayout.addView(drawGraphic);
        }
    }

    private boolean hasPermission() {
        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) {
            return checkSelfPermission(PERMISSION_CAMERA) == PackageManager.PERMISSION_GRANTED;
        } else {
            return true;
        }
    }

    private void requestPermission() {
        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) {
            if (shouldShowRequestPermissionRationale(PERMISSION_CAMERA)) {
                Toast.makeText(
                        this,
                        "该演示需要获取相机权限",
                        Toast.LENGTH_LONG)
                        .show();
            }
            requestPermissions(new String[]{PERMISSION_CAMERA}, PERMISSIONS_REQUEST);
        }
    }

    @Override
    public void onRequestPermissionsResult(
            final int requestCode, final String[] permissions, final int[] grantResults) {
        super.onRequestPermissionsResult(requestCode, permissions, grantResults);
        if (requestCode == PERMISSIONS_REQUEST) {
            if (allPermissionsGranted(grantResults)) {
                startCamera();
            } else {
                requestPermission();
            }
        }
    }

    private static boolean allPermissionsGranted(final int[] grantResults) {
        for (int result : grantResults) {
            if (result != PackageManager.PERMISSION_GRANTED) {
                return false;
            }
        }
        return true;
    }
}

请问是什么原因导致boundingBox出现轻微偏移？如需补充相关信息我会及时提供。

问题原因及修复方案

导致边界框偏移的核心问题有4个，对应修复方式如下：

矩阵重复应用：DrawGraphic的onDraw方法同时调用setMatrix和concat两次应用同一个转换矩阵，相当于坐标被重复转换两次直接导致偏移，删除其中一行即可，仅保留canvas.concat(getMappingMatrix(imageProxy, previewView))。
必要参数未传递：MainActivity的getObjectResults方法创建DrawGraphic实例时，没有传入imageProxy和previewView，导致转换矩阵计算时两个核心参数为空，坐标转换逻辑完全失效。需要修改getObjectResults的入参，把检测回调里拿到的imageProxy和全局previewView透传进去，构造DrawGraphic时传入对应参数。
坐标参数顺序错误：Canvas.drawRect的入参顺序为left, top, right, bottom，现有代码传入的是rect.left, rect.bottom, rect.right, rect.top，上下坐标颠倒也会导致框的位置翻转偏移，调整参数顺序即可。
预览和分析分辨率不匹配：ImageAnalysis设置的目标分辨率为640*480，而CameraX的Preview输出会根据手机屏幕比例自适应裁剪，二者宽高比不一致会导致坐标映射偏差。可以给Preview.Builder也设置和ImageAnalysis相同的目标分辨率，或者将previewView的缩放模式设置为FIT_CENTER保证裁剪逻辑一致：
```
previewView.setScaleType(PreviewView.ScaleType.FIT_CENTER);
```

内容的提问来源于stack exchange，提问作者RunMildew

如何修复Android端Google ML+CameraX目标检测boundingBox坐标偏移问题

DrawGraphic.java #

MainActivity.java #

问题原因及修复方案 #

DrawGraphic.java

MainActivity.java

问题原因及修复方案