基于Siamese网络与Triplet Loss的人脸识别模型训练故障求助

问题概述 #

使用Triplet Loss结合Siamese网络构建人脸识别算法时，出现损失值无法降至margin以下、任意两张图片的L2距离几乎为0的问题，尝试过4种网络结构（含ResNet50）、调整多种学习率、添加Dropout正则化均未解决。

核心问题排查与解决方案 #

1. 数据预处理缺失 #

神经网络对输入尺度极度敏感，原代码未做归一化处理，且图片缩放逻辑冗余：

• 修复代码：

if img.shape == (112, 112, 3):
    rgb_image = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
    resized_im = cv2.resize(rgb_image, (64, 64))  # 直接指定目标尺寸，避免冗余计算
    resized_im = resized_im / 255.0  # 归一化到[0,1]区间
    lst_imgs.append(resized_im)

• 额外修正：原代码通过range(iterations)遍历文件夹，存在字符串转数字的潜在错误，改为直接遍历文件夹名称：

person_folders = os.listdir(dataset_path)
for folder_name in person_folders: 
    current_path = os.path.join(dataset_path, folder_name)
    # 后续读取图片逻辑不变

2. 三元组生成逻辑错误 #

原代码生成大量重复无效样本（如anchor与positive为同一张图），且效率极低：

• 正确三元组生成逻辑：每个anchor从同一人中选不同图片作为positive，从其他人群中选图片作为negative：

import random

anchors = []
positives = []
negatives = []

person_ids = list(imgs.keys())

for main_id in person_ids:
    person_imgs = imgs[main_id]
    # 遍历当前人的每张图作为anchor，排除最后一张留作候选positive
    for idx, anchor_img in enumerate(person_imgs[:-1]):
        # 从当前人的剩余图片中选positive，避免与anchor重复
        positive_candidates = person_imgs[:idx] + person_imgs[idx+1:]
        positive_img = random.choice(positive_candidates)
        
        # 从其他人群中随机选negative
        negative_person_ids = [pid for pid in person_ids if pid != main_id]
        negative_person_id = random.choice(negative_person_ids)
        negative_img = random.choice(imgs[negative_person_id])
        
        anchors.append(anchor_img)
        positives.append(positive_img)
        negatives.append(negative_img)

3. Triplet Loss实现错误 #

原模型输出为拼接后的向量，但损失函数错误地将其当作三个embedding的列表处理：

• 方案1：拆分拼接向量

def triplet_loss(y_true, y_pred, alpha=0.2):
    embedding_dim = 128  # 与模型输出embedding维度一致
    anchor = y_pred[:, :embedding_dim]
    positive = y_pred[:, embedding_dim:2*embedding_dim]
    negative = y_pred[:, 2*embedding_dim:]

    pos_dist = tf.reduce_sum(tf.square(anchor - positive), axis=-1)
    neg_dist = tf.reduce_sum(tf.square(anchor - negative), axis=-1)

    loss = tf.maximum(pos_dist - neg_dist + alpha, 0.0)
    return tf.reduce_mean(loss)

• 方案2：模型直接返回三个embedding列表（更简洁）

# 模型定义时取消拼接，直接返回三个embedding
siamese_model = Model(
    inputs=[input_anchor, input_positive, input_negative],
    outputs=[embedding_anchor, embedding_positive, embedding_negative]
)
# 损失函数保持原写法即可（此时y_pred是三个embedding的列表）

4. 模型结构优化 #

基础模型问题修复

原模型全连接层过大易导致梯度消失/过拟合，且缺少Triplet Loss必需的L2归一化：

def create_siamese_network(input_shape):
    input_image = Input(shape=input_shape)
    x = Conv2D(64, (3, 3), activation='relu')(input_image)
    x = MaxPooling2D()(x)
    x = Conv2D(128, (3, 3), activation='relu')(x)
    x = MaxPooling2D()(x)
    x = Conv2D(256, (3, 3), activation='relu')(x)
    x = MaxPooling2D()(x)
    x = Flatten()(x)
    x = Dense(512, activation='relu')(x)
    x = Dense(128, activation='relu')(x)
    # 强制embedding在单位球面上，保证距离计算有效性
    output = tf.keras.layers.Lambda(lambda x: tf.math.l2_normalize(x, axis=1))(x)
    return Model(inputs=input_image, outputs=output)

ResNet50模型修复

原代码存在结构混乱、未做预训练模型输入预处理的问题：

def create_resnet_siamese(input_shape):
    base_model = tf.keras.applications.ResNet50(weights='imagenet', include_top=False, input_shape=input_shape)
    # 冻结前120层，仅训练后期层
    for layer in base_model.layers[:120]:
        layer.trainable = False
    for layer in base_model.layers[120:]:
        layer.trainable = True
    
    x = base_model.output
    x = tf.keras.layers.GlobalAveragePooling2D()(x)  # 替代Flatten减少参数，避免过拟合
    x = tf.keras.layers.Dense(512, activation='relu')(x)
    x = tf.keras.layers.Dense(128, activation='relu')(x)
    output = tf.keras.layers.Lambda(lambda x: tf.math.l2_normalize(x, axis=1))(x)
    
    model = Model(inputs=base_model.input, outputs=output)
    return model

# 构建完整Siamese模型
siamese_network = create_resnet_siamese(input_shape)
input_anchor = Input(shape=input_shape)
input_positive = Input(shape=input_shape)
input_negative = Input(shape=input_shape)

# 应用ResNet专属输入预处理
anchor_preprocessed = tf.keras.applications.resnet50.preprocess_input(input_anchor)
positive_preprocessed = tf.keras.applications.resnet50.preprocess_input(input_positive)
negative_preprocessed = tf.keras.applications.resnet50.preprocess_input(input_negative)

embedding_anchor = siamese_network(anchor_preprocessed)
embedding_positive = siamese_network(positive_preprocessed)
embedding_negative = siamese_network(negative_preprocessed)

siamese_model = Model(
    inputs=[input_anchor, input_positive, input_negative],
    outputs=[embedding_anchor, embedding_positive, embedding_negative]
)

5. 训练策略调整 #

• 学习率：初始用1e-4，配合学习率衰减策略，避免过小学习率导致训练停滞
• 批次大小：建议用32/64，平衡训练稳定性与效率
• 验证集：划分验证集监控训练状态，避免过拟合

from sklearn.model_selection import train_test_split
from tensorflow.keras.callbacks import ReduceLROnPlateau

# 划分训练/验证集
anchors_train, anchors_val, positives_train, positives_val, negatives_train, negatives_val = train_test_split(
    anchors, positives, negatives, test_size=0.2, random_state=42
)

# 学习率衰减回调
lr_scheduler = ReduceLROnPlateau(monitor='loss', factor=0.5, patience=5, min_lr=1e-6)

# 编译与训练
siamese_model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=1e-4), loss=triplet_loss)
history = siamese_model.fit(
    [anchors_train, positives_train, negatives_train],
    y=None,  # Triplet Loss无需真实标签，传None即可
    epochs=50,
    batch_size=32,
    validation_data=([anchors_val, positives_val, negatives_val], None),
    callbacks=[lr_scheduler]
)

6. 验证步骤 #

• 训练前手动计算随机三元组的原始距离，确认样本合理性
• 训练初期用PCA降维可视化embedding分布，查看是否出现区分度
• 定期统计anchor-positive、anchor-negative的距离分布，验证训练趋势

内容的提问来源于stack exchange，提问作者kareem said