使用AlexNet分类音频频谱图时遇矩阵形状不匹配错误求助

尝试使用AlexNet对3秒音频片段生成的频谱图进行分类，已知AlexNet要求输入图像尺寸为224×224，已对训练和测试数据集做相应变换，但运行时出现如下错误：

RuntimeError: Mat1 and mat2 shapes cannot be multiplied (256x65536 and 1024x4096)

附上相关代码，寻求问题排查与解决方法：

数据变换代码 #

data_transform_train = transforms.Compose([
        transforms.Resize(256),
        transforms.CenterCrop(256),
        transforms.ToTensor(),
        transforms.Normalize(norm_mean_train, norm_std_train),
    ])

创建数据加载器代码 #

train_size = int(len(train_data_df))
test_size = int(len(test_data_df))

ins_dataset_train = Audio(
    df=train_data_df[:train_size],
    transform=data_transform_train,
)

ins_dataset_test = Audio(
    df=test_data_df[:test_size],
    transform=data_transform_test,
)

train_loader = torch.utils.data.DataLoader(
    ins_dataset_train,
    batch_size=256,
    shuffle=True
)

test_loader = torch.utils.data.DataLoader(
    ins_dataset_test,
    batch_size=256,
    shuffle=True
)

AlexNet模型代码 #

class AlexNet(nn.Module):
    def __init__(self, output_dim):
        super().__init__()

        self.features = nn.Sequential(
            nn.Conv2d(3, 64, 3, 2, 1),  # in_channels, out_channels, kernel_size, stride, padding
            nn.MaxPool2d(2),  # kernel_size
            nn.ReLU(inplace=True),
            nn.Conv2d(64, 192, 3, padding=1),
            nn.MaxPool2d(2),
            nn.ReLU(inplace=True),
            nn.Conv2d(192, 384, 3, padding=1),
            nn.ReLU(inplace=True),
            nn.Conv2d(384, 256, 3, padding=1),
            nn.ReLU(inplace=True),
            nn.Conv2d(256, 256, 3, padding=1),
            nn.MaxPool2d(2),
            nn.ReLU(inplace=True)
        )

        self.classifier = nn.Sequential(
            nn.Dropout(0.5),
            nn.Linear(256 * 2 * 2, 4096),
            nn.ReLU(inplace=True),
            nn.Dropout(0.5),
            nn.Linear(4096, 4096),
            nn.ReLU(inplace=True),
            nn.Linear(4096, output_dim),
        )

    def forward(self, x):
        x = self.features(x)
        h = x.view(x.shape[0], -1)
        x = self.classifier(h)
        return x, h

output_dim = 2
model = AlexNet(output_dim)

def count_parameters(model):
    return sum(p.numel() for p in model.parameters() if p.requires_grad)

print(f'The model has {count_parameters(model):,} trainable parameters')

def initialize_parameters(m):
    if isinstance(m, nn.Conv2d):
        nn.init.kaiming_normal_(m.weight.data, nonlinearity='relu')
        nn.init.constant_(m.bias.data, 0)
    elif isinstance(m, nn.Linear):
        nn.init.xavier_normal_(m.weight.data, gain=nn.init.calculate_gain('relu'))
        nn.init.constant_(m.bias.data, 0)

model.apply(initialize_parameters)

学习率查找器代码 #

class LRFinder:
    
    def __init__(self, model, optimizer, criterion, device):
        
        self.optimizer = optimizer
        self.model = model
        self.criterion = criterion
        self.device = device
        torch.save(model.state_dict(), 'init_params.pt')

    def range_test(self, iterator, end_lr=10, num_iter=100, smooth_f=0.05, diverge_th=5):

        lrs = []
        losses = []
        best_loss = float('inf')

        lr_scheduler = ExponentialLR(self.optimizer, end_lr, num_iter)
        iterator = IteratorWrapper(iterator)

        for iteration in range(num_iter):

            loss = self._train_batch(iterator)
            lrs.append(lr_scheduler.get_last_lr()[0])
            lr_scheduler.step()

            if iteration > 0:
                loss = smooth_f * loss + (1 - smooth_f) * losses[-1]

            if loss < best_loss:
                best_loss = loss

            losses.append(loss)

            if loss > diverge_th * best_loss:
                print("Stopping early, the loss has diverged")
                break

        model.load_state_dict(torch.load('init_params.pt'))
        return lrs, losses

    def _train_batch(self, iterator):

        self.model.train()
        self.optimizer.zero_grad()
        x, y = iterator.get_batch()
        x = x.to(self.device)
        y = y.to(self.device)
        y_pred, _ = self.model(x)
        loss = self.criterion(y_pred, y)
        loss.backward()
        self.optimizer.step()

        return loss.item()

from torch.optim.lr_scheduler import _LRScheduler

class ExponentialLR(_LRScheduler):
    def __init__(self, optimizer, end_lr, num_iter, last_epoch=-1):
        self.end_lr = end_lr
        self.num_iter = num_iter
        super(ExponentialLR, self).__init__(optimizer, last_epoch)

    def get_lr(self):
        curr_iter = self.last_epoch
        r = curr_iter / self.num_iter
        return [base_lr * (self.end_lr / base_lr) ** r
                for base_lr in self.base_lrs]

class IteratorWrapper:
    
    def __init__(self, iterator):
        self.iterator = iterator
        self._iterator = iter(iterator)

    def __next__(self):
        try:
            inputs, labels = next(self._iterator)
        except StopIteration:
            self._iterator = iter(self.iterator)
            inputs, labels, *_ = next(self._iterator)
        return inputs, labels

    def get_batch(self):
        return next(self)

start_learning_rate = 1e-7
optimizer = optim.Adam(model.parameters(), lr=start_learning_rate)
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
criterion = nn.CrossEntropyLoss()
model = model.to(device)
criterion = criterion.to(device)

END_LR = 10
NUM_ITER = 100

lr_finder = LRFinder(model, optimizer, criterion, device)
lrs, losses = lr_finder.range_test(train_loader, END_LR, NUM_ITER)

问题根源与解决方法 #

核心问题 #

错误提示的矩阵维度不匹配，本质是特征提取模块输出的特征图展平后维度，与分类器第一层全连接层的预设输入维度完全不匹配：

• 你的输入图像实际是256×256（数据变换里CenterCrop(256)），经过特征提取模块的卷积和池化后，最终展平的维度是256*16*16=65536（对应错误中的256x65536，第一个256是batch size）。
• 但你在分类器里硬编码了第一层全连接层的输入维度为256*2*2=1024，导致矩阵乘法时维度冲突。

分步解决 #

1. 修正数据变换到AlexNet要求的224×224

把数据变换中的CenterCrop(256)改为CenterCrop(224)，同时同步修改测试集的变换：

data_transform_train = transforms.Compose([
        transforms.Resize(256),
        transforms.CenterCrop(224),  # 修正为224×224
        transforms.ToTensor(),
        transforms.Normalize(norm_mean_train, norm_std_train),
    ])

2. 修正分类器全连接层的输入维度

不要硬编码特征维度，推荐两种方式：

方式一：动态计算特征维度（最稳妥）

修改模型定义，在初始化时用dummy tensor自动计算特征提取后的展平维度：

class AlexNet(nn.Module):
    def __init__(self, output_dim):
        super().__init__()

        self.features = nn.Sequential(
            nn.Conv2d(3, 64, 3, 2, 1),
            nn.MaxPool2d(2),
            nn.ReLU(inplace=True),
            nn.Conv2d(64, 192, 3, padding=1),
            nn.MaxPool2d(2),
            nn.ReLU(inplace=True),
            nn.Conv2d(192, 384, 3, padding=1),
            nn.ReLU(inplace=True),
            nn.Conv2d(384, 256, 3, padding=1),
            nn.ReLU(inplace=True),
            nn.Conv2d(256, 256, 3, padding=1),
            nn.MaxPool2d(2),
            nn.ReLU(inplace=True)
        )

        # 用dummy tensor计算特征展平后的维度
        with torch.no_grad():
            dummy_input = torch.randn(1, 3, 224, 224)
            feat_output = self.features(dummy_input)
            self.feature_flatten_dim = feat_output.flatten(1).shape[1]

        self.classifier = nn.Sequential(
            nn.Dropout(0.5),
            nn.Linear(self.feature_flatten_dim, 4096),  # 使用动态计算的维度
            nn.ReLU(inplace=True),
            nn.Dropout(0.5),
            nn.Linear(4096, 4096),
            nn.ReLU(inplace=True),
            nn.Linear(4096, output_dim),
        )

    def forward(self, x):
        x = self.features(x)
        h = x.flatten(1)
        x = self.classifier(h)
        return x, h

方式二：手动计算后硬编码

如果输入是224×224，按你的特征提取模块计算最终特征图尺寸：

• 初始输入：3×224×224
• 第一层Conv+Pool后：64×56×56
• 第二层Conv+Pool后：192×28×28
• 三次Conv后保持维度不变，最后一次Pool后：256×14×14
• 展平后维度：2561414=50176

修改分类器第一层：

nn.Linear(50176, 4096),

3. 可选：调整batch size

你的batch size设为256，若GPU内存不足，可调小至32/64，避免出现OOM错误。

内容的提问来源于stack exchange，提问作者csStudent2102