如何基于含人脸与人物标注的数据集为Caffe创建LMDB数据库？

Hey there! I get that setting up LMDBs in Caffe can feel overwhelming at first, so let's walk through this clearly—starting with the train.txt file you're confused about, then moving to the full LMDB creation process.

First: Understand What train.txt Is #

Each line in train.txt follows this format:

relative/path/to/image.jpg 0

• The first part is the relative path to your image (relative to the root folder where all your images are stored).
• The number after the space is your class label: you can assign numbers to your two classes however you want (e.g., 0 for "face" and 1 for "person", or vice versa). Just make sure you're consistent—this number will map to the output layer of your Caffe model later.

How to Generate train.txt from OpenImages CSV #

Since you have the OpenImages CSV with bounding box annotations, we need to filter out only the face/person classes and map each image to its correct label. Here's a simple Python script to do this (adjust the paths and class names to match your dataset):

import csv

# Define your classes and their labels (match the exact class names from your CSV)
CLASS_MAPPING = {
    "Face": 0,
    "Person": 1
}

# Path to your OpenImages annotation CSV
ANNOTATION_CSV = "path/to/your/annotations.csv"
# Output path for the final train.txt
OUTPUT_TXT = "train.txt"
# Root folder where your downloaded images are stored
IMAGE_ROOT = "path/to/your/images/folder"

# Avoid duplicate entries for images with multiple annotations
processed_images = set()

with open(OUTPUT_TXT, "w") as txt_file:
    with open(ANNOTATION_CSV, "r") as csv_file:
        reader = csv.DictReader(csv_file)
        for row in reader:
            class_name = row["LabelName"]  # Adjust to match your CSV's class column header
            image_filename = f"{row['ImageID']}.jpg"  # OpenImages uses ImageID as the default filename
            
            # Skip irrelevant classes and already processed images
            if class_name not in CLASS_MAPPING or image_filename in processed_images:
                continue
            
            # Write the relative path and corresponding label
            txt_file.write(f"{image_filename} {CLASS_MAPPING[class_name]}\n")
            processed_images.add(image_filename)

Note: Double-check your CSV's column names (like LabelName or ImageID)—OpenImages may have slightly different headers depending on the version you downloaded. If an image has both face and person annotations, this script adds it once using the first matching class. Adjust the logic if you need to handle multi-label cases.

Full Steps to Create the LMDB #

Once you have train.txt ready, follow these steps to generate the LMDB:

1. Locate the convert_imageset Tool #

Caffe includes a built-in tool called convert_imageset, which lives in the build/tools folder after you compile Caffe. Make sure you've compiled Caffe successfully before proceeding.

2. Run the convert_imageset Command #

Open your terminal and execute this command (replace placeholders with your actual paths):

/path/to/caffe/build/tools/convert_imageset \
    --resize_height=224 \  # Optional: Resize images to a fixed height (match your model's input size)
    --resize_width=224 \   # Optional: Resize images to a fixed width
    --shuffle \            # Optional: Shuffle image order (helps with training generalization)
    /path/to/your/images/folder/ \  # Root folder of your images (matches IMAGE_ROOT from the script)
    /path/to/your/train.txt \       # Path to your generated train.txt
    /path/to/save/train_lmdb        # Output path for your LMDB

• --resize_height/--resize_width: Most Caffe models expect fixed-size inputs (e.g., 224x224 for AlexNet/VGG), so use these flags to standardize your images.
• --shuffle: This mixes up the image order to prevent your model from learning spurious patterns based on data sequence.

3. Verify the LMDB #

After the command finishes, you’ll see two files (data.mdb and lock.mdb) in your output train_lmdb folder. To confirm it’s valid, generate a mean file (used for training normalization) with Caffe's compute_image_mean tool:

/path/to/caffe/build/tools/compute_image_mean \
    /path/to/save/train_lmdb \
    /path/to/save/mean.binaryproto

This mean file will be referenced in your Caffe model's prototxt and solver files later.

Quick Tips #

• Double-check that all paths in train.txt are correct—Caffe will throw an error if it can’t find an image.
• Repeat the same process with a validation subset to create val.txt and val_lmdb for evaluating your model.
• Keep track of your label mapping (e.g., 0=Face, 1=Person)—you’ll need to set your model’s final layer to output 2 classes, and use this mapping to interpret results.

内容的提问来源于stack exchange，提问作者Piyal George