生成TFRecord遇absl.flags未识别'mode' flag及Flag不打印问题求助

核心问题分析 #

1. 函数重名冲突：代码同时导入numpy.split和自定义split函数，导致main函数实际调用的是numpy的split而非自定义函数，引发逻辑错误的同时间接干扰flag解析流程。
2. 未处理未知命令行flag：tf.compat.v1.app.run()会自动解析命令行参数，若运行环境中存在未定义的mode flag（如依赖模块自动添加或误传参数），则触发UnrecognizedFlagError。
3. 全局变量与flag混淆：代码同时定义全局变量和同名flag，但未将全局变量值同步到flag，导致flag打印为空。

修复步骤与代码 #

1. 修复函数重名 #

将自定义split函数重命名为split_dataset，避免与numpy.split冲突。

2. 忽略未知flag #

添加flags.FLAGS.allow_unknown_flags = True，跳过未定义的mode flag解析。

3. 统一使用flag（或同步全局变量到flag） #

直接将全局变量值赋值给flag，简化测试流程；也可通过命令行传入参数替代此操作。

修复后的完整代码 #

from __future__ import division
from __future__ import print_function
from __future__ import absolute_import

import os
import io
import pandas as pd
import tensorflow as tf
from PIL import Image
from object_detection.utils import dataset_util
from collections import namedtuple
from numpy import split

# 全局路径配置
csv_input = "C:\\Users\\Documents\\Research\\ShortCut\\Model_B\\PTrain_labels.csv"
output_path = "C:\\Users\\Documents\\OutPutPath\\Output.tfrecord"
image_dir = "C:\\Users\\Documents\\Research\\ShortCut\\Model_B\\Base"

flags = tf.compat.v1.flags
flags.DEFINE_string('csv_input', '', 'Path to the CSV input')
flags.DEFINE_string('output_path', '', 'Path to output TFRecord')
flags.DEFINE_string('image_dir', '', 'Path to images')
# 允许忽略未知命令行flag，解决'mode'报错
flags.FLAGS.allow_unknown_flags = True
FLAGS = flags.FLAGS

# 将全局变量值同步到flag（方便测试，也可通过命令行传入）
FLAGS.csv_input = csv_input
FLAGS.output_path = output_path
FLAGS.image_dir = image_dir

print("csv_input flag:", FLAGS.csv_input)
print("output_path flag:", FLAGS.output_path)
print("image_dir flag:", FLAGS.image_dir)


def class_text_to_int(row_label):
    if row_label == "M":
        return 1
    elif row_label == "J":
        return 2
    else:
        return None

# 重命名自定义split函数，避免与numpy.split冲突
def split_dataset(df, group):
    data = namedtuple('data', ['filename', 'object'])
    gb = df.groupby(group)
    return [data(filename, gb.get_group(x)) for filename, x in zip(gb.groups.keys(), gb.groups)]

def create_tf_example(group, path):
    with tf.io.gfile.GFile(os.path.join(path, group.filename), 'rb') as fid:
        encoded_jpg = fid.read()
    encoded_jpg_io = io.BytesIO(encoded_jpg)
    image = Image.open(encoded_jpg_io)
    width, height = image.size

    filename = group.filename.encode('utf8')
    image_format = b'jpg'
    xmins = []
    xmaxs = []
    ymins = []
    ymaxs = []
    classes_text = []
    classes = []

    for _, row in group.object.iterrows():
        xmins.append(row['xmin'] / width)
        xmaxs.append(row['xmax'] / width)
        ymins.append(row['ymin'] / height)
        ymaxs.append(row['ymax'] / height)
        classes_text.append(row['class'].encode('utf8'))
        classes.append(class_text_to_int(row['class']))

    tf_example = tf.train.Example(features=tf.train.Features(feature={
       'image/height': dataset_util.int64_feature(height),
       'image/width': dataset_util.int64_feature(width),
       'image/filename': dataset_util.bytes_feature(filename),
       'image/source_id': dataset_util.bytes_feature(filename),
       'image/encoded': dataset_util.bytes_feature(encoded_jpg),
       'image/format': dataset_util.bytes_feature(image_format),
       'image/object/bbox/xmin': dataset_util.float_list_feature(xmins),
       'image/object/bbox/xmax': dataset_util.float_list_feature(xmaxs),
       'image/object/bbox/ymin': dataset_util.float_list_feature(ymins),
       'image/object/bbox/ymax': dataset_util.float_list_feature(ymaxs),
       'image/object/class/text': dataset_util.bytes_list_feature(classes_text),
       'image/object/class/label': dataset_util.int64_list_feature(classes),
    }))
    return tf_example

def main(_):
    # Print the values of csv_input, output_path, and image_dir
    print("csv_input:", csv_input)
    print("output_path:", output_path)
    print("image_dir:", image_dir)

    print("csv_input:", FLAGS.csv_input)
    print("output_path:", FLAGS.output_path)
    print("image_dir:", FLAGS.image_dir)

    writer = tf.io.TFRecordWriter(FLAGS.output_path)
    path = FLAGS.image_dir
    examples = pd.read_csv(FLAGS.csv_input)
    # 调用修复后的split_dataset函数
    grouped = split_dataset(examples, 'filename')
    for group in grouped:
        tf_example = create_tf_example(group, path)
        writer.write(tf_example.SerializeToString())

    print("Writing TFRecord to:", FLAGS.output_path)

    writer.close()
    print('Successfully created the TFRecords: {}'.format(FLAGS.output_path))

if __name__ == '__main__':
    try:
        tf.compat.v1.app.run()
    except Exception as e:
        print("An error occurred:", str(e))

额外说明 #

• 若希望通过命令行传入参数，可移除FLAGS.csv_input = csv_input等赋值语句，运行时使用：

  python your_script.py --csv_input "C:\\...\\PTrain_labels.csv" --output_path "C:\\...\\Output.tfrecord" --image_dir "C:\\...\\Base"
  

• 函数重名是隐藏的逻辑错误，即使未触发flag报错，也会导致分组逻辑失效，必须修复。

内容的提问来源于stack exchange，提问作者Juan Carlos Rubio Polania