Android Jetpack Compose中ML-Kit姿态检测坐标与预览匹配问题

基于Jetpack Compose开发Android应用时，使用ML-Kit姿态检测的关键点坐标直接绘制会显示在屏幕左上角，对应缩小版的640x480处理图像。尝试用预览尺寸（1080x2125）缩放坐标后，位置仍然不正确。

相关分析器代码：

class PoseAnalyzer(
    private val onSuccess: (result: AnalyzeResultData) -> Unit,
): ImageAnalysis.Analyzer {
    private val options = PoseDetectorOptions.Builder()
        .setDetectorMode(PoseDetectorOptions.STREAM_MODE)
        .build()

    private val poseDetector = PoseDetection.getClient(options)

    @ExperimentalGetImage
    override fun analyze(imageProxy: ImageProxy) {
        val mediaImage = imageProxy.image
        if (mediaImage != null) {
            val rotationDegrees = imageProxy.imageInfo.rotationDegrees
            val processImage = InputImage.fromMediaImage(mediaImage, rotationDegrees)
            poseDetector.process(processImage)
                .addOnSuccessListener { pose ->
                    val result = AnalyzeResultData(pose, processImage.toSizeData())
                    onSuccess(result)
                }
                .addOnFailureListener {
                }
                .addOnCompleteListener {
                    imageProxy.close()
                }
        }
    }
}

尝试的缩放代码：

private fun scaleAndGetOffset(
        point: SizeData,
        analyzedImage: SizeData,
        screenSize: SizeData,
        cameraSelector: CameraSelector,
    ): Offset {
        val rateX = screenSize.x/analyzedImage.x
        val rateY = screenSize.y/analyzedImage.y
        return Offset(
            x = if (cameraSelector == CameraSelector.DEFAULT_BACK_CAMERA) point.x * rateX else screenSize.x-point.x * rateX,
            y = point.y * rateY
        )
}

核心问题 #

直接用屏幕尺寸缩放忽略了三个关键因素：预览画面的黑边适配、图像旋转的坐标转换、前后摄像头的镜像逻辑与旋转的结合。

解决方案步骤 #

1. 传递图像旋转参数 #

修改分析器，将图像旋转角度传递到外部，用于后续坐标转换：

// 调整数据类，添加旋转角度字段
data class AnalyzeResultData(val pose: Pose, val analyzedImageSize: SizeData, val rotationDegrees: Int)

// 更新分析器的成功回调逻辑
override fun analyze(imageProxy: ImageProxy) {
    val mediaImage = imageProxy.image
    if (mediaImage != null) {
        val rotationDegrees = imageProxy.imageInfo.rotationDegrees
        val processImage = InputImage.fromMediaImage(mediaImage, rotationDegrees)
        poseDetector.process(processImage)
            .addOnSuccessListener { pose ->
                val result = AnalyzeResultData(pose, processImage.toSizeData(), rotationDegrees)
                onSuccess(result)
            }
            .addOnFailureListener { }
            .addOnCompleteListener { imageProxy.close() }
    }
}

2. 获取预览视图的实际尺寸 #

在Compose中获取PreviewView的实际显示尺寸（而非直接用屏幕尺寸），因为预览可能因宽高比不匹配存在黑边：

// 示例：在Compose中获取PreviewView尺寸
val context = LocalContext.current
val previewView = remember { PreviewView(context) }
val previewSize by rememberUpdatedState(
    previewView.drawable?.intrinsicSize ?: Size(0f, 0f)
)

3. 修正坐标缩放与转换逻辑 #

重新实现缩放函数，处理宽高比适配、旋转转换、镜像逻辑：

private fun scaleAndGetOffset(
    point: SizeData,
    analyzedImage: SizeData,
    previewSize: Size,
    cameraSelector: CameraSelector,
    imageRotation: Int
): Offset {
    // 分析图像的实际宽高（根据旋转角度调整）
    val analyzedWidth = if (imageRotation % 180 == 0) analyzedImage.x else analyzedImage.y
    val analyzedHeight = if (imageRotation % 180 == 0) analyzedImage.y else analyzedImage.x

    // 计算预览与分析图像的宽高比
    val previewRatio = previewSize.width / previewSize.height
    val analyzedRatio = analyzedWidth / analyzedHeight

    var scaleX: Float
    var scaleY: Float
    var offsetX = 0f
    var offsetY = 0f

    // 按宽高比适配，避免拉伸
    if (previewRatio > analyzedRatio) {
        // 预览更宽，按高度适配，左右留黑边
        scaleY = previewSize.height / analyzedHeight
        scaleX = scaleY
        offsetX = (previewSize.width - analyzedWidth * scaleX) / 2
    } else {
        // 预览更高，按宽度适配，上下留黑边
        scaleX = previewSize.width / analyzedWidth
        scaleY = scaleX
        offsetY = (previewSize.height - analyzedHeight * scaleY) / 2
    }

    // 将ML Kit的坐标转换为预览方向的坐标
    val (rotatedX, rotatedY) = when (imageRotation) {
        90 -> Pair(analyzedHeight - point.y, point.x)
        180 -> Pair(analyzedWidth - point.x, analyzedHeight - point.y)
        270 -> Pair(point.y, analyzedWidth - point.x)
        else -> Pair(point.x, point.y) // 0度
    }

    // 处理前后摄像头镜像
    val finalX = if (cameraSelector == CameraSelector.DEFAULT_FRONT_CAMERA) {
        previewSize.width - (rotatedX * scaleX + offsetX)
    } else {
        rotatedX * scaleX + offsetX
    }
    val finalY = rotatedY * scaleY + offsetY

    return Offset(finalX, finalY)
}

4. 调用修正后的函数 #

在Compose绘制时，传入正确的参数调用缩放函数：

// 示例：绘制关键点
Canvas(modifier = Modifier.fillMaxSize()) {
    result?.pose?.allPoseLandmarks?.forEach { landmark ->
        val point = SizeData(landmark.x, landmark.y)
        val offset = scaleAndGetOffset(
            point = point,
            analyzedImage = result.analyzedImageSize,
            previewSize = previewSize,
            cameraSelector = cameraSelector,
            imageRotation = result.rotationDegrees
        )
        drawCircle(
            color = Color.Red,
            radius = 10f,
            center = offset
        )
    }
}

关键说明 #

• 必须基于PreviewView的实际尺寸计算，而非屏幕尺寸，否则黑边区域会导致坐标偏移。
• ML Kit返回的坐标是基于旋转后的输入图像，需先将坐标转换为预览画面的方向，再进行缩放。
• 前后摄像头的镜像处理要在旋转转换之后进行，避免方向错乱。

内容的提问来源于stack exchange，提问作者isifzade