Hey there! Based on your requirement—fully offline browser-side face detection + recognition, with each device only needing to identify one specific face—I’ve got a solid solution that avoids backend dependencies and builds on reliable web tech. Let’s dive in:

TensorFlow.js is the perfect fit here because it runs entirely in the browser, no Node.js server required. We’ll combine two key models to cover detection and recognition, plus local storage to save the user’s face template for offline use.

Step 1: Face Detection with BlazeFace #

BlazeFace is a super lightweight, fast face detection model designed for mobile/web. It’s way more efficient than tracking.js for real-time detection, and it gives you precise bounding boxes to crop faces for recognition.

Step 2: Face Feature Extraction with MobileFaceNet #

MobileFaceNet is a compact model that converts faces into 128-dimensional feature vectors (embeddings). These vectors are unique to each person, so we can compare them to check for matches. Unlike face-recognition.js, this runs 100% client-side.

Step 3: Single-Face Training & Recognition Workflow #

Since each device only needs to recognize one face, the workflow is straightforward:

• Initial Setup: When the user first uses the app, ask them to take a clear photo of their face.
  1. Use BlazeFace to detect and crop the face from the image.
  2. Run the cropped face through MobileFaceNet to get its feature vector.
  3. Store this vector in localStorage or IndexedDB (IndexedDB is better for larger data, but 128 floats are tiny).
• Recognition: For subsequent uses:
  1. Detect the face from the camera feed using BlazeFace.
  2. Extract its feature vector with MobileFaceNet.
  3. Calculate the cosine similarity between the new vector and the stored one. If the similarity is above a threshold (like 0.6-0.7, adjust based on testing), it’s a match.

Quick Code Snippet to Get Started #

First, make sure you’ve downloaded the BlazeFace and MobileFaceNet model files to your local project (so they load offline):

// Load models from local paths (replace with your actual file paths)
async function loadModels() {
  const detector = await tf.loadGraphModel('/models/blazeface/model.json');
  const recognizer = await tf.loadLayersModel('/models/mobilefacenet/model.json');
  return { detector, recognizer };
}

// Extract face embedding
async function getFaceEmbedding(image, detector, recognizer) {
  // Detect faces
  const predictions = await detector.predict(image);
  if (predictions.length === 0) return null;
  
  // Crop the first detected face (adjust coordinates as needed)
  const [x1, y1, x2, y2] = predictions[0].bbox;
  const height = image.shape[0];
  const width = image.shape[1];
  const croppedFace = tf.image.cropAndResize(
    image.expandDims(0),
    [[y1/height, x1/width, y2/height, x2/width]],
    [0],
    [112, 112] // MobileFaceNet expects 112x112 input
  );
  
  // Normalize pixel values to [-1, 1]
  const normalized = croppedFace.div(127.5).sub(1);
  
  // Get embedding
  const embedding = await recognizer.predict(normalized).data();
  tf.dispose([predictions, croppedFace, normalized]); // Clean up tensors to save memory
  return Array.from(embedding);
}

// Calculate cosine similarity between two embeddings
function cosineSimilarity(vecA, vecB) {
  let dotProduct = 0;
  let normA = 0;
  let normB = 0;
  for (let i = 0; i < vecA.length; i++) {
    dotProduct += vecA[i] * vecB[i];
    normA += vecA[i] * vecA[i];
    normB += vecB[i] * vecB[i];
  }
  return dotProduct / (Math.sqrt(normA) * Math.sqrt(normB));
}

Why This Beats Your Previous Attempts #

• tracking.js: Only handles detection—no way to generate unique face embeddings for recognition. TensorFlow.js fills that gap with purpose-built feature extraction models.
• face-recognition.js: Requires a Node.js backend for core processing, but TensorFlow.js runs everything client-side, making it ideal for offline scenarios.

Offline Optimization Tips #

• Host Models Locally: Download the model files (BlazeFace and MobileFaceNet) and include them in your project bundle so they don’t need to be fetched from the internet.
• Model Quantization: Use quantized versions of the models to reduce file size and improve inference speed (TensorFlow.js supports this out of the box).
• WebGL Acceleration: TensorFlow.js automatically uses WebGL if available, which drastically speeds up model processing on most devices.

内容的提问来源于stack exchange，提问作者MarkZ