如何基于SceneKit/ARKit从相机视角渲染场景深度图？

1. 配置SCNTechnique的JSON定义 #

SCNTechnique依赖JSON定义渲染流程，核心是新增深度渲染通道，从相机视角输出深度信息。创建DepthTechnique.json文件，内容如下：

{
  "passes": {
    "depthPass": {
      "draw": "DRAW_SCENE",
      "program": "depthProgram",
      "clear": true,
      "clearColor": [0.0, 0.0, 0.0, 1.0],
      "depthAttachment": {
        "pixelFormat": "RGBA32Float"
      },
      "colorAttachments": {
        "color": {
          "pixelFormat": "RGBA32Float",
          "clearColor": [0.0, 0.0, 0.0, 1.0]
        }
      }
    },
    "finalPass": {
      "draw": "DRAW_QUAD",
      "program": "copyProgram",
      "colorAttachments": {
        "color": {
          "texture": "depthPassColor"
        }
      }
    }
  },
  "programs": {
    "depthProgram": {
      "vertexShader": "depthVertexShader.glsl",
      "fragmentShader": "depthFragmentShader.glsl"
    },
    "copyProgram": {
      "vertexShader": "copyVertexShader.glsl",
      "fragmentShader": "copyFragmentShader.glsl"
    }
  },
  "sequence": ["depthPass", "finalPass"],
  "symbols": {}
}

2. 编写GLSL着色器文件 #

深度渲染顶点着色器（depthVertexShader.glsl） #

attribute vec4 position;
attribute vec3 normal;

uniform mat4 modelViewProjectionMatrix;
uniform mat4 modelViewMatrix;

varying vec4 v_position;

void main() {
    v_position = modelViewMatrix * position;
    gl_Position = modelViewProjectionMatrix * position;
}

深度渲染片段着色器（depthFragmentShader.glsl） #

varying vec4 v_position;
uniform float zNear;
uniform float zFar;

float linearizeDepth(float depth) {
    return (2.0 * zNear) / (zFar + zNear - depth * (zFar - zNear));
}

void main() {
    float depth = -v_position.z / v_position.w;
    float linearDepth = linearizeDepth(depth);
    gl_FragColor = vec4(linearDepth, linearDepth, linearDepth, 1.0);
}

复制用顶点着色器（copyVertexShader.glsl） #

attribute vec4 position;
attribute vec2 texcoord;

varying vec2 v_texcoord;

void main() {
    v_texcoord = texcoord;
    gl_Position = position;
}

复制用片段着色器（copyFragmentShader.glsl） #

varying vec2 v_texcoord;
uniform sampler2D color;

void main() {
    gl_FragColor = texture2D(color, v_texcoord);
}

3. 代码中加载并应用SCNTechnique #

在SceneKit视图控制器中完成资源加载与绑定：

// 加载technique配置文件
guard let techniqueURL = Bundle.main.url(forResource: "DepthTechnique", withExtension: "json") else {
    fatalError("深度渲染配置文件不存在")
}
guard let technique = SCNTechnique(contentsOf: techniqueURL) else {
    fatalError("加载深度渲染配置失败")
}

// 配置相机参数（根据你的场景调整zNear/zFar）
let cameraNode = SCNNode()
let camera = SCNCamera()
camera.zNear = 0.1
camera.zFar = 100.0
cameraNode.camera = camera
scene.rootNode.addChildNode(cameraNode)

// 传递相机参数给technique
technique.setValue(NSNumber(value: camera.zNear), forKey: "zNear")
technique.setValue(NSNumber(value: camera.zFar), forKey: "zFar")

// 将technique应用到场景
sceneView.scene?.technique = technique

4. 获取并处理渲染后的深度图 #

通过SCNSceneRendererDelegate获取深度纹理并转换为可使用的图像：

func renderer(_ renderer: SCNSceneRenderer, didRenderScene scene: SCNScene, atTime time: TimeInterval) {
    guard let technique = sceneView.scene?.technique else { return }
    guard let depthTexture = technique.value(forKey: "depthPassColor") as? MTLTexture else { return }
    
    // 将Metal纹理转换为UIImage
    let ciImage = CIImage(mtlTexture: depthTexture, options: nil)
    let context = CIContext(options: [.useSoftwareRenderer: false])
    if let cgImage = context.createCGImage(ciImage!, from: ciImage!.extent) {
        let depthImage = UIImage(cgImage: cgImage)
        // 在这里处理或保存深度图
    }
}

关键注意事项 #

• 线性化深度：默认深度值是透视非线性的，着色器中的linearizeDepth函数将其转换为0（近平面）到1（远平面）的线性灰度值，对应你需要的效果。
• 纹理精度：使用RGBA32Float格式保证深度值精度，避免8位纹理带来的精度丢失。
• 相机参数：必须准确设置zNear和zFar，否则线性化后的深度值会出现偏差。

内容的提问来源于stack exchange，提问作者Andy Hin