基于ARCore实现跨设备用户交互及远程渲染的技术问询

Great question—this is one of the most fundamental concepts to wrap your head around when building shared AR experiences with ARCore! Let me break down exactly how this works:

• The core idea: Distributed rendering + state synchronization
  You’re right that each device handles its own camera feed and local rendering entirely on its own. Remote users don’t "send rendered objects" to your screen—instead, they send lightweight, critical AR data that your device uses to render the same object in the correct shared space.

• What data gets exchanged between devices?
  When a user adds an object (like your arrow), their device sends over:

  1. The ARCore Anchor data: This ties the object to a fixed point in the shared real-world space. Anchors are key because they adjust as ARCore refines its understanding of the environment, keeping objects stable.
  2. Object metadata: The type of object (arrow, cube, etc.), its position, rotation, and scale relative to the anchor.
  3. Optional session context: Things like detected plane data, to help both devices stay aligned on the shared world coordinate system.

• How your device uses this data
  When your device receives the data, it does two main things:

  1. Creates a matching ARCore Anchor in its own local AR session, using the shared anchor coordinates to align it with the real world.
  2. Renders the object (arrow) locally, attaching it to the newly created anchor. Since both devices are using the same anchor reference point relative to the real world, the object appears in the exact same physical location for both users.

• A quick example to make it concrete
  Suppose I add an arrow on my phone, pointing at a coffee table. My device binds that arrow to an anchor located on the table’s surface. It sends the anchor’s world position, plus the arrow’s rotation/scale, to your device. Your phone receives this, creates an anchor at the same real-world spot (using ARCore’s tracking to find that table), then draws the arrow on top of it. Now we both see the arrow pointing at the table—no remote rendering required.

• One critical note: Spatial alignment
  For this to work seamlessly, both devices need to agree on the same shared world coordinate system. This is often done using ARCore’s Cloud Anchors (which sync anchor positions across devices via Google’s cloud) or by having both users scan the same physical space to establish a common reference frame. Without this alignment, objects would appear in wrong positions for one or both users.

内容的提问来源于stack exchange，提问作者bhavin shah