TensorFlow元组数据集过滤报错：TypeError参数不匹配求助

错误原因 #

当你的数据集元素是((image_data, caption_data), target_data)这种嵌套元组结构时，TensorFlow的filter方法会自动将这个嵌套结构展开成两个独立参数传入过滤函数：第一个参数是(image_data, caption_data)，第二个是target_data，但你定义的filter_funct只接收一个参数，因此触发了参数不匹配的TypeError。

解决办法（两种可选） #

方法1：修改过滤函数的参数定义，匹配展开后的结构 #

直接让函数接收两个参数，对应展开后的两部分数据：

import tensorflow as tf

# Create dummy data
image_data = [tf.constant([1]), tf.constant([2]), tf.constant([3])]
caption_data = [tf.constant([10]), tf.constant([20]), tf.constant([30])]
target_data = [tf.constant([100]), tf.constant([200]), tf.constant([300])]

# Create a dataset from the dummy data
dataset = tf.data.Dataset.from_tensor_slices(((image_data, caption_data), target_data))

# 修改参数为两个：(image_tensor, caption_tensor) 和 target_tensor
def filter_funct(image_caption_tuple, target_tensor):
    (image_tensor, caption_tensor) = image_caption_tuple
    return target_tensor > 150

# Apply the filter function
filtered_dataset = dataset.filter(filter_funct)

# Print the filtered dataset
for ((image_tensor, caption_tensor), target_tensor) in filtered_dataset:
    print("Image Tensor:", image_tensor.numpy())
    print("Caption Tensor:", caption_tensor.numpy())
    print("Target Tensor:", target_tensor.numpy())

方法2：用@tf.function装饰过滤函数，保留原参数结构 #

通过装饰器让TensorFlow正确识别嵌套元组结构，不自动展开：

import tensorflow as tf

# Create dummy data
image_data = [tf.constant([1]), tf.constant([2]), tf.constant([3])]
caption_data = [tf.constant([10]), tf.constant([20]), tf.constant([30])]
target_data = [tf.constant([100]), tf.constant([200]), tf.constant([300])]

# Create a dataset from the dummy data
dataset = tf.data.Dataset.from_tensor_slices(((image_data, caption_data), target_data))

# 添加@tf.function装饰器
@tf.function
def filter_funct(data):
    ((image_tensor, caption_tensor), target_tensor) = data
    return target_tensor > 150

# Apply the filter function
filtered_dataset = dataset.filter(filter_funct)

# Print the filtered dataset
for ((image_tensor, caption_tensor), target_tensor) in filtered_dataset:
    print("Image Tensor:", image_tensor.numpy())
    print("Caption Tensor:", caption_tensor.numpy())
    print("Target Tensor:", target_tensor.numpy())

针对筛选损坏图像需求的扩展 #

如果要判断图像是否损坏，可以在过滤函数中添加图像有效性检查，比如判断图像张量的形状是否合法、是否包含NaN/Inf值：

@tf.function
def filter_corrupted_images(data):
    ((image_tensor, caption_tensor), target_tensor) = data
    # 检查图像形状是否符合预期（比如至少是HWC格式）
    valid_shape = tf.rank(image_tensor) == 3
    # 检查图像是否有非法值
    valid_values = tf.reduce_all(tf.math.is_finite(image_tensor))
    # 结合你的目标条件
    return valid_shape & valid_values & (target_tensor > 150)

内容的提问来源于stack exchange，提问作者SoraHeart