Android平台ARCore中如何实现SurfaceView内特定对象的触摸选择与交互？

Hey there! Let's break down how to solve both of your problems, since you want to ditch radio buttons and implement direct object selection via touch. This all boils down to hit detection—checking which object your touch event intersects with—and managing object states to control rendering and interactions.

In ARCore, you can't just check screen coordinates directly because objects exist in 3D world space, not just on the screen. Here's the step-by-step process:

• Capture the touch event: Override onTouchEvent in your AR activity or custom view to grab the screen X/Y coordinates when the user taps.
• Convert screen touch to a 3D ray: Use ARCore's Camera to get the view-projection matrix, then convert the screen point into a ray that starts at the camera's position and extends into the world.
• Check for object intersections: For each of your 3D objects, run a ray vs. bounding box (or collision shape) test to see if the ray hits the object. Pick the closest hit (since multiple objects might be under the same screen point).
• Mark the selected object: Keep track of which object was hit, and update its state to "selected".

Example Code Snippet #

@Override
public boolean onTouchEvent(MotionEvent event) {
    if (event.getAction() == MotionEvent.ACTION_DOWN) {
        Frame frame = arSession.update();
        Camera camera = frame.getCamera();
        
        // Convert screen coordinates to Normalized Device Coordinates (NDC)
        float ndcX = (event.getX() / getWidth()) * 2 - 1;
        float ndcY = (event.getY() / getHeight()) * -2 + 1;
        
        // Get view and projection matrices from ARCore's camera
        float[] viewMatrix = new float[16];
        camera.getViewMatrix(viewMatrix, 0);
        float[] projMatrix = new float[16];
        camera.getProjectionMatrix(projMatrix, 0, 0.1f, 100f);
        
        // Calculate ray origin (camera position) and direction
        float[] rayOrigin = new float[3];
        float[] rayDirection = new float[3];
        Matrix.invertM(viewMatrix, 0, viewMatrix, 0);
        Matrix.multiplyMV(rayOrigin, 0, viewMatrix, 0, new float[]{ndcX, ndcY, -1, 1}, 0);
        Matrix.multiplyMV(rayDirection, 0, viewMatrix, 0, new float[]{ndcX, ndcY, 1, 0}, 0);
        Vector3.normalize(rayDirection, rayDirection);
        
        // Check each object for ray intersection
        MyARObject selectedObj = null;
        float closestDistance = Float.MAX_VALUE;
        for (MyARObject obj : arSceneObjects) {
            float hitDistance = obj.getBoundingBox().intersectRay(rayOrigin, rayDirection);
            // intersectRay returns -1 if no hit, else the distance from camera to hit point
            if (hitDistance != -1 && hitDistance < closestDistance) {
                closestDistance = hitDistance;
                selectedObj = obj;
            }
        }
        
        if (selectedObj != null) {
            // Update selection state: only one object selected at a time
            for (MyARObject obj : arSceneObjects) {
                obj.setSelected(obj == selectedObj);
            }
            // Handle any selection logic here (e.g., show a highlight)
        }
        return true;
    }
    return super.onTouchEvent(event);
}

Notes #

• You'll need a BoundingBox class for your MyARObject that implements intersectRay to check if the ray hits the box. For simple objects, an Axis-Aligned Bounding Box (AABB) is fast and sufficient.
• For complex models, you can use more precise collision shapes (like OBB or triangle-level detection), but this will impact performance.

For a SurfaceView (whether 2D or 3D), the core idea is similar: detect which object the user touched, then use that selection to control rendering and rotation. Here's how to implement it:

Step 1: Capture Touch and Detect Hit #

Override onTouchEvent in your SurfaceView to check which object is under the touch point. For 2D objects, this is just a simple bounds check; for 3D, use the same ray detection logic as in ARCore.

Step 2: Manage Selected Object State #

Keep a reference to the currently selected object in your renderer or view. Only this object will have rotation enabled, and you can adjust rendering (e.g., highlight it) if needed.

Step 3: Control Rendering and Rotation #

In your renderer's onDrawFrame method, apply rotation only to the selected object. All objects can still be rendered, but only the selected one gets the rotation update.

Example Code Snippets #

SurfaceView Touch Handling

public class MySurfaceView extends GLSurfaceView {
    private MyRenderer renderer;
    private MyObject obj1, obj2;

    public MySurfaceView(Context context) {
        super(context);
        renderer = new MyRenderer();
        setRenderer(renderer);
        
        // Initialize your two objects
        obj1 = new MyObject(100, 100, 200, 200); // x, y, width, height (2D example)
        obj2 = new MyObject(300, 100, 200, 200);
        renderer.setObjects(obj1, obj2);
    }

    @Override
    public boolean onTouchEvent(MotionEvent event) {
        if (event.getAction() == MotionEvent.ACTION_DOWN) {
            float x = event.getX();
            // Invert Y for OpenGL (since OpenGL's Y starts at the bottom of the screen)
            float y = getHeight() - event.getY();
            
            MyObject hitObj = null;
            if (obj1.getBounds().contains(x, y)) {
                hitObj = obj1;
            } else if (obj2.getBounds().contains(x, y)) {
                hitObj = obj2;
            }
            
            renderer.setSelectedObject(hitObj);
            return true;
        }
        return super.onTouchEvent(event);
    }
}

Renderer with Rotation Control

public class MyRenderer implements GLSurfaceView.Renderer {
    private MyObject obj1, obj2;
    private MyObject selectedObject;

    public void setObjects(MyObject obj1, MyObject obj2) {
        this.obj1 = obj1;
        this.obj2 = obj2;
    }

    public void setSelectedObject(MyObject obj) {
        this.selectedObject = obj;
    }

    @Override
    public void onDrawFrame(GL10 gl) {
        GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT | GLES20.GL_DEPTH_BUFFER_BIT);
        
        // Render all objects
        obj1.draw();
        obj2.draw();
        
        // Only rotate the selected object
        if (selectedObject != null) {
            selectedObject.rotate(0.5f); // Adjust rotation speed as needed
        }
    }

    // ... other renderer methods (onSurfaceCreated, onSurfaceChanged)
}

Notes #

• For 3D objects in SurfaceView, use the same ray detection logic from the ARCore section—just replace ARCore's camera with your own perspective camera.
• If you want to toggle selection (tap to select, tap again to deselect), add a check: if the tapped object is already selected, set selectedObject to null.

内容的提问来源于stack exchange，提问作者Shubham Sejpal