TIMM库中的DropPath实现是否仅为普通Dropout而非DropPath或DropConnect？

结论 #

你提到的判断不成立，这段代码是标准的DropPath（随机深度）实现，既不是普通Dropout，也不是DropConnect。

具体辨析 #

• 和普通Dropout的差异：普通Dropout会对输入张量的每一个元素独立生成随机掩码，随机丢弃单个神经元的激活值；而这段代码生成的掩码维度为(x.shape[0],) + (1,) * (x.ndim - 1)，仅和batch维度匹配，同一个batch样本的所有特征维度共享同一个掩码，相当于对每个样本的整个残差路径做整体保留或丢弃，这正是DropPath的核心逻辑，常被用在残差网络中减少过拟合。
• 和DropConnect的差异：正统的DropConnect是对网络层的权重参数做随机失活，而非对激活值路径做丢弃，这段代码的处理对象是输入的激活张量x，不是网络权重，所以也不属于DropConnect。代码注释里作者也明确提到了早期命名存在歧义，所以特意将该实现改名为drop_path，避免和正统的DropConnect概念混淆。

对应实现代码 #

def drop_path(x, drop_prob: float = 0., training: bool = False):
    """Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).
    This is the same as the DropConnect impl I created for EfficientNet, etc networks, however,
    the original name is misleading as 'Drop Connect' is a different form of dropout in a separate paper...
    I've opted for
    changing the layer and argument names to 'drop path' rather than mix DropConnect as a layer name and use
    'survival rate' as the argument.
    """
    if drop_prob == 0. or not training:
        return x
    keep_prob = 1 - drop_prob
    shape = (x.shape[0],) + (1,) * (x.ndim - 1)  # work with diff dim tensors, not just 2D ConvNets
    random_tensor = keep_prob + torch.rand(shape, dtype=x.dtype, device=x.device)
    random_tensor.floor_()  # binarize
    output = x.div(keep_prob) * random_tensor
    return output

内容的提问来源于stack exchange，提问作者Ink