BRFv4+ThreeJS双人脸眼镜佩戴失效及自定义模型问题咨询

Hey there, let’s break down how to solve your two issues—getting glasses to show up on both tracked faces and building your own custom eyewear models.

Issue 1: Glasses Only Appear on One Face #

You’ve already set numFacesToTrack = 2 and updated brfManager.setNumFacesToTrack(numFacesToTrack), which is the first step. Let’s dig into why the second face isn’t getting glasses:

1. Verify BRFv4 is Tracking Both Faces #

First, confirm BRFv4 is actually detecting and tracking two faces. Add a quick debug log in your update loop to check face states:

for(var i = 0; i < faces.length; i++) {
  var face = faces[i];
  console.log(`Face ${i} state: ${brfv4.BRFState[face.state]}`); // Add this debug line
  if(face.state === brfv4.BRFState.FACE_TRACKING) {
    // ... existing code
  }
}

If one face shows FACE_DETECTION instead of FACE_TRACKING, adjust your BRF tracking parameters to be more lenient:

// Tweak these values to allow easier tracking of multiple faces
brfManager.setFaceTrackingStartParams(
  maxFaceSize * 0.15,  // Lower minimum face size to catch smaller faces
  maxFaceSize * 1.00,
  24,                  // Reduce minimum points required to start tracking
  25,                  // Lower confidence threshold
  24
);

2. Ensure ThreeJS Models Are Loaded for Both Faces #

Check if your t3d instance is actually loading two model instances. Add a log after loading:

function loadModels() {
  if(t3d) {
    t3d.removeAll();
    t3d.loadOcclusionHead("assets/brfv4_occlusion_head.json", numFacesToTrack);
    t3d.loadModel("assets/brfv4_model.json", numFacesToTrack);
    console.log(`Total models loaded: ${t3d._models.length}`); // Verify count (adjust property name if needed)
  }
}

If the count is 1 instead of 2, double-check that the BRFv4 ThreeJS helper (t3d) supports multiple model instances for numFacesToTrack > 1. Some default implementations might require explicit cloning of models for each face index.

3. Confirm Model Update Binds to the Correct Face #

Make sure t3d.update(i, face, true) is correctly mapping the face index i to the corresponding model instance. If the helper uses a single model for all faces, you’ll need to modify it to clone the model for each tracked face.

Issue 2: Creating & Loading Custom Glasses Models #

Building your own eyewear involves 3 main steps: modeling, exporting, and integrating with BRFv4 + ThreeJS.

1. Model the Glasses #

• Use a 3D tool like Blender to create your glasses. Align the model to BRFv4’s 68 facial landmarks:
  • Position the frames to match the eye landmarks (points 36-45 in BRFv4’s vertex list).
  • Center the model at the origin (0,0,0) so it aligns with the face’s center point.
  • Keep the model low-poly for real-time performance.

2. Export for ThreeJS #

• In Blender, use the built-in ThreeJS Exporter add-on (enable it in Blender’s preferences).
• Export your model as a ThreeJS JSON file:
  • Uncheck unnecessary options (like animations, if not needed).
  • Ensure "Vertices" and "Faces" are enabled.
  • Set the scale to match BRFv4’s coordinate system (1 unit ≈ 1 pixel in your video feed, adjust as needed).

3. Load the Custom Model #

Replace the default model path with your custom JSON file in loadModels:

t3d.loadModel("assets/custom_glasses.json", numFacesToTrack);

If the model is misaligned after loading:

• Adjust the model’s position/scale/rotation in your 3D tool and re-export.
• Or add post-load adjustments in your code:

  // After loading, tweak model properties for face index i
  var customGlasses = t3d._models[i];
  customGlasses.scale.set(1.2, 1.2, 1.2); // Scale up slightly
  customGlasses.position.y += 10; // Shift down to align with eyes
  

Final Notes #

• Test with well-lit, clear faces to ensure BRFv4 can track both reliably.
• If the t3d helper has limited multi-face support, extend it to clone models for each tracked face index.

内容的提问来源于stack exchange，提问作者Asif Khan