PyTorch中shuffle=False时每epoch图像顺序仍变化的问题
问题原因及解决办法
核心问题出在你使用的SubsetRandomSampler上:
- 这个采样器的设计逻辑就是每个epoch都会随机打乱传入的索引列表,以此实现随机采样。
- 另外,PyTorch的
DataLoader有个规则:当手动指定了sampler参数时,shuffle参数会被直接忽略,所以你设置的shuffle=False根本没起作用。
你尝试添加generator参数无效,是因为SubsetRandomSampler的随机逻辑不依赖这个全局生成器,它内部有自己的随机机制,就算固定种子,它还是会在每个epoch重新打乱索引,只是打乱的结果固定,而非保持顺序不变。
两种解决思路
1. 改用Subset创建固定子集(推荐)
直接通过Subset类生成固定的训练/测试子集,此时shuffle=False会正常生效,每个epoch的样本顺序完全一致:
def create_loader_from_data_set(data_set, n_samples, batch_size, num_workers, test_size=0.2): indices = list(range(len(data_set))) selected_indices = random.sample(indices, n_samples) train_indices, test_indices = train_test_split(selected_indices, test_size=test_size, random_state=42) # 创建固定子集,而非用采样器动态采样 train_subset = torch.utils.data.Subset(data_set, train_indices) test_subset = torch.utils.data.Subset(data_set, test_indices) # 此时shuffle=True/False会正常控制是否打乱顺序 train_loader = DataLoader(train_subset, batch_size=batch_size, num_workers=num_workers, shuffle=False) test_loader = DataLoader(test_subset, batch_size=batch_size, num_workers=num_workers, shuffle=False) return train_loader, test_loader
2. 替换为SequentialSampler
如果你坚持要用采样器模式,把SubsetRandomSampler换成SequentialSampler,它会严格按照传入的索引顺序遍历,不会打乱:
def create_loader_from_data_set(data_set, n_samples, batch_size, num_workers, test_size=0.2): indices = list(range(len(data_set))) selected_indices = random.sample(indices, n_samples) train_indices, test_indices = train_test_split(selected_indices, test_size=test_size, random_state=42) # 用SequentialSampler按顺序遍历索引 train_sampler = torch.utils.data.SequentialSampler(train_indices) test_sampler = torch.utils.data.SequentialSampler(test_indices) train_loader = DataLoader(data_set, batch_size=batch_size, num_workers=num_workers, sampler=train_sampler) test_loader = DataLoader(data_set, batch_size=batch_size, num_workers=num_workers, sampler=test_sampler) return train_loader, test_loader
内容的提问来源于stack exchange,提问作者mor hale
相关产品推荐
相关产品推荐

