PyTorch全局剪枝固化后保存模型体积未较原模型减小的问题
PyTorch全局剪枝后保存模型无体积压缩的异常问题
问题场景
基于PyTorch对深度学习模型执行全局非结构化剪枝操作时,出现如下异常:
- 基础参数:原始未剪枝的模型大小约为77.5MB
- 操作流程:按标准流程完成全局非结构化剪枝、逐层校验各剪枝层稀疏度、确认全局稀疏度达标,调用
prune.remove()将剪枝效果永久固化后,使用torch.save保存模型状态字典 - 异常表现:保存得到的剪枝后模型文件体积和原始未剪枝模型完全一致,未出现预期的压缩效果。现有如下完整实现代码,希望定位该问题的根本原因。
关联实现代码
import torch.nn.utils.prune as prune parameters_to_prune = ( (model.encoder[0], 'weight'), (model.up_conv1[0], 'weight'), (model.up_conv2[0], 'weight'), (model.up_conv3[0], 'weight'), ) print(parameters_to_prune) prune.global_unstructured( parameters_to_prune, pruning_method=prune.L1Unstructured, amount=0.2, ) print( "Sparsity in Encoder.weight: {:.2f}%".format( 100. * float(torch.sum(model.encoder[0].weight == 0)) / float(model.encoder[0].weight.nelement()) ) ) print( "Sparsity in up_conv1.weight: {:.2f}%".format( 100. * float(torch.sum(model.up_conv1[0].weight == 0)) / float(model.up_conv1[0].weight.nelement()) ) ) print( "Sparsity in up_conv2.weight: {:.2f}%".format( 100. * float(torch.sum(model.up_conv2[0].weight == 0)) / float(model.up_conv2[0].weight.nelement()) ) ) print( "Sparsity in up_conv3.weight: {:.2f}%".format( 100. * float(torch.sum(model.up_conv3[0].weight == 0)) / float(model.up_conv3[0].weight.nelement()) ) ) print( "Global sparsity: {:.2f}%".format( 100. * float( torch.sum(model.encoder[0].weight == 0) + torch.sum(model.up_conv1[0].weight == 0) + torch.sum(model.up_conv2[0].weight == 0) + torch.sum(model.up_conv3[0].weight == 0) ) / float( model.encoder[0].weight.nelement() + model.up_conv1[0].weight.nelement() + model.up_conv2[0].weight.nelement() + model.up_conv3[0].weight.nelement() ) ) ) # 剪枝效果永久固化 prune.remove(model.encoder[0], "weight") prune.remove(model.up_conv1[0], "weight") prune.remove(model.up_conv2[0], "weight") prune.remove(model.up_conv3[0], "weight") # 保存模型 PATH = "C:\\PrunedNet.pt" torch.save(model.state_dict(), PATH)
内容的提问来源于stack exchange,提问作者Whatsinthename
相关产品推荐
相关产品推荐

