PyTorch中损失函数出现NaN的解决方法咨询
PyTorch中损失函数出现NaN的解决方法咨询
我在PyTorch里定义的损失函数包含1/x项和其他几个项,我的神经网络最后一层是sigmoid激活层,所以输出值在0到1之间。
训练过程中,某个输入到1/x的值肯定变得极小了,因为我的损失值变成了这样:
loss: 11.047459 [729600/235474375]
loss: 9.348356 [731200/235474375]
loss: 7.184393 [732800/235474375]
loss: 8.699876 [734400/235474375]
loss: 7.178806 [736000/235474375]
loss: 8.090066 [737600/235474375]
loss: 12.415799 [739200/235474375]
loss: 10.422441 [740800/235474375]
loss: 8.335846 [742400/235474375]
loss: nan [744000/235474375]
loss: nan [745600/235474375]
loss: nan [747200/235474375]
loss: nan [748800/235474375]
loss: nan [750400/235474375]
我想知道有没有办法在遇到NaN时“回退”训练,或者重新定义损失函数来避免NaN的出现?谢谢!
备注:内容来源于stack exchange,提问作者Mike
相关产品推荐
相关产品推荐

