目标检测指标技术问询：IoU、过拟合及API扩展相关问题

Hey there! Great work getting your custom dataset training and evaluation set up with the TensorFlow Object Detection API and TensorBoard—weighted PASCAL metrics are a solid choice for initial evaluation. Let’s walk through each of your questions clearly:

1. Does an un-detected annotated object affect IoU? #

IoU (Intersection over Union) is calculated only for paired ground truth and prediction boxes. If an annotated target isn’t detected at all (no prediction box matches it), that target doesn’t contribute to any IoU calculation. That said, it will hurt your overall mAP (mean Average Precision), since mAP accounts for recall of all ground truth objects.

2. Does a false negative (annotated target not detected) affect IoU? #

This is essentially the same scenario as question 1! False negatives mean no matching prediction exists for a ground truth box, so they don’t impact IoU values directly. However, false negatives reduce your recall rate (since recall = TP/(TP+FN)), which in turn pulls down your mAP—since mAP is a combination of precision and recall across all confidence thresholds.

3. What impact do false positives in un-annotated areas have on evaluation metrics? #

False positives (FPs) are predictions where no ground truth object exists, and they hit your metrics in two key ways:

• Precision drops: Precision is calculated as TP/(TP+FP)—more FPs increase the denominator, lowering precision.
• mAP decreases: mAP is the area under the precision-recall curve for each class. Lower precision at every recall threshold shrinks this area, dragging down overall mAP.
  In PASCAL VOC’s evaluation logic specifically, each FP counts as a "mistake" that reduces the precision score at the corresponding confidence threshold, which directly impacts the AP calculation for that class.

4. Has anyone modified the Object Detection API to add metrics like accuracy, TP/FP/TN/FN? Can you share code or guidance? #

Absolutely— the API is designed to be modular, so extending evaluation logic is straightforward. Here’s a practical way to add TP/FP/FN tracking (and derived precision/recall) by modifying the pascal_voc_evaluator.py in the object_detection/metrics/ directory:

Step 1: Initialize counters in the evaluator class #

Add these lines to the __init__ method to track per-class counts:

from collections import defaultdict

# Track TP/FP/FN per class
self.tp_counts = defaultdict(int)
self.fp_counts = defaultdict(int)
self.fn_counts = defaultdict(int)

Step 2: Update counts during per-image evaluation #

In the _evaluate_detection_single_image method, after matching ground truth and prediction boxes:

# Track matched ground truth indices
matched_gt_indices = set()
for match in matches:
    gt_label = match.groundtruth_label
    self.tp_counts[gt_label] += 1
    matched_gt_indices.add(match.groundtruth_index)

# Count FNs: ground truth boxes with no matching prediction
for gt_idx in range(len(groundtruth_boxes)):
    if gt_idx not in matched_gt_indices:
        gt_label = groundtruth_labels[gt_idx]
        self.fn_counts[gt_label] += 1

# Count FPs: predictions with no matching ground truth
all_pred_indices = set(range(len(detection_boxes)))
matched_pred_indices = set(m.match_index for m in matches)
for pred_idx in all_pred_indices - matched_pred_indices:
    pred_label = detection_classes[pred_idx]
    self.fp_counts[pred_label] += 1

Step 3: Compute and return new metrics #

In the _compute_metrics method, calculate precision/recall for each class and add them to the metrics dict:

metrics = super()._compute_metrics()  # Keep original PASCAL metrics

for label in self.tp_counts.keys():
    tp = self.tp_counts[label]
    fp = self.fp_counts[label]
    fn = self.fn_counts[label]
    
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0.0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0.0
    
    metrics[f'precision/class_{label}'] = precision
    metrics[f'recall/class_{label}'] = recall
    metrics[f'tp/class_{label}'] = tp
    metrics[f'fp/class_{label}'] = fp
    metrics[f'fn/class_{label}'] = fn

# Note: Accuracy is rarely used in object detection because TN (true negatives) are massive
# (most of the image is background), so accuracy scores are misleadingly high

After making these changes, update your pipeline config to use this modified evaluator, and you’ll see the new metrics in TensorBoard. You can also create a custom evaluator class inheriting from BaseEvaluator if you want to keep the original code untouched.

5. If I use 30% of my training data for validation to monitor overfitting, what indicators show overfitting? #

Overfitting happens when your model learns noise in the training data instead of generalizable patterns. Watch for these signs:

• mAP gap: Training set mAP keeps rising, but validation mAP plateaus or drops—especially if the gap between training and validation mAP is large (e.g., training mAP = 0.92, validation = 0.7).
• IoU discrepancy: Average IoU on the training set is much higher than on validation.
• Loss curves: Training loss continues to decrease, but validation loss stops dropping and starts increasing, or stays consistently higher than training loss with a growing gap.
• Precision/Recall imbalance: Training precision/recall are near perfect, but validation precision/recall are significantly lower.
• Visual examples: In TensorBoard, training images show nearly perfect detections, but validation images have frequent false positives, false negatives, or misaligned prediction boxes.

内容的提问来源于stack exchange，提问作者user8574993