Colab中GPU运行神经网络网格搜索过慢，是否存在使用问题？

一、GPU使用正确性验证 #

先确认GPU是否正常工作，在代码开头添加以下代码：

print(f"CUDA可用: {torch.cuda.is_available()}")
print(f"当前GPU: {torch.cuda.get_device_name(0)}")

训练时可打印模型所在设备，确认模型已加载到GPU：

model = SimpleNN(input_dim, param['hidden_dim1'], param['hidden_dim2'], param['dropout_rate']).to('cuda')
print(f"模型设备: {next(model.parameters()).device}")

你的代码中存在冗余操作：evaluate_model里重复对已在GPU上的张量调用.to('cuda')，虽不会报错但会增加不必要开销，建议直接移除。

二、代码中的严重逻辑错误 #

1. 变量名冲突
   grid_search函数参数名为params，但循环中又用params遍历参数网格，会覆盖外部传入的参数，导致逻辑混乱。修改如下：

def grid_search(city, start_year, end_year, param_grid):  # 修改参数名为param_grid
    # ... 其他代码
    for param in ParameterGrid(param_grid):  # 循环变量改为param
        model = SimpleNN(input_dim, param['hidden_dim1'], param['hidden_dim2'], param['dropout_rate']).to('cuda')
        # ... 其余代码同步替换为param

2. 训练/测试数据混淆
   你的代码用测试集标签作为训练标签，且用训练特征评估测试标签，完全违背机器学习基本逻辑，既得不到有效模型，也造成无效训练开销。修正数据加载函数：

def load_and_preprocess_data(city, start_year, end_year):
    train_file = f'{city}{start_year}.csv'
    test_file = f'{city}{end_year}.csv'

    train_data = pd.read_csv(train_file, header=None)
    test_data = pd.read_csv(test_file, header=None)

    # 假设数据集第一行为标签，后续为特征
    x_train = (train_data.iloc[1:, :] > 0).astype(np.float32).values
    y_train = (train_data.iloc[0, :] > 0).astype(np.float32).values
    x_test = (test_data.iloc[1:, :] > 0).astype(np.float32).values
    y_test = (test_data.iloc[0, :] > 0).astype(np.float32).values

    return x_train, y_train, x_test, y_test

同步修改grid_search中的数据处理逻辑，用训练集标签训练，测试集评估：

def grid_search(city, start_year, end_year, param_grid):
    x_train, y_train, x_test, y_test = load_and_preprocess_data(city, start_year, end_year)
    input_dim = x_train.shape[0]

    # 先在CPU创建张量，再按需移至GPU
    x_train_tensor = torch.tensor(x_train).float().T
    y_train_tensor = torch.tensor(y_train).float().reshape(-1, 1)
    x_test_tensor = torch.tensor(x_test).float().T
    y_test_tensor = torch.tensor(y_test).float().reshape(-1, 1)
    
    # 创建训练数据集与DataLoader，添加pin_memory加速CPU到GPU传输
    train_dataset = torch.utils.data.TensorDataset(x_train_tensor, y_train_tensor)
    train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=64, shuffle=True, pin_memory=True)

    best_score = float('inf')
    best_params = None

    for param in ParameterGrid(param_grid):
        model = SimpleNN(input_dim, param['hidden_dim1'], param['hidden_dim2'], param['dropout_rate']).to('cuda')
        
        criterion = nn.BCELoss()
        optimizer = optim.Adam(model.parameters(), lr=param['learning_rate'])

        train_model(model, criterion, optimizer, train_loader, param['epochs'])

        # 用测试集评估模型泛化能力
        metrics = evaluate_model(model, x_test_tensor, y_test_tensor)
        if metrics['mse'] < best_score:
            best_score = metrics['mse']
            best_params = param
        
        # 清理GPU内存，避免内存泄漏
        del model
        torch.cuda.empty_cache()

    return best_params, best_score

3. 损失函数与任务匹配优化
   你的任务是二分类，模型输出为sigmoid后的0-1值，训练用BCELoss合理，但评估用MSELoss不如用准确率或BCEWithLogitsLoss（可移除模型最后的sigmoid，数值稳定性更好）：

# 修改模型，移除最后一层sigmoid
class SimpleNN(nn.Module):
    def __init__(self, input_dim, hidden_dim1, hidden_dim2, dropout_rate):
        super(SimpleNN, self).__init__()
        self.fc1 = nn.Linear(input_dim, hidden_dim1)
        self.fc2 = nn.Linear(hidden_dim1, hidden_dim2)
        self.fc3 = nn.Linear(hidden_dim2, 1)
        self.dropout = nn.Dropout(dropout_rate)

    def forward(self, x):
        x = torch.relu(self.fc1(x))
        x = self.dropout(x)
        x = torch.relu(self.fc2(x))
        x = self.dropout(x)
        return x  # 直接返回logits，交给损失函数处理

# 训练时使用BCEWithLogitsLoss
criterion = nn.BCEWithLogitsLoss()

三、网格搜索速度优化 #

你的参数网格共有33545=900个组合，每个组合最多训练100个epoch，这是速度慢的核心原因，可从以下几点优化：

1. 减少参数组合数
   先缩小参数范围，比如epochs先只试10,30，learning_rate保留0.001,0.01,0.1，dropout_rate保留0.1,0.3,0.5，组合数可降至162个，速度提升5倍。优先对影响大的参数（如学习率、隐藏层维度）做搜索，其余参数先固定合理值，找到最优后再细化。

2. 添加早停机制
   监控验证集损失，当损失连续N个epoch不下降时提前停止训练，避免无效epoch：

def train_model(model, criterion, optimizer, train_loader, val_loader, epochs, patience=5):
    model.train()
    best_val_loss = float('inf')
    counter = 0
    for epoch in range(epochs):
        # 训练步骤
        train_loss = 0.0
        for x_batch, y_batch in train_loader:
            x_batch, y_batch = x_batch.to('cuda', non_blocking=True), y_batch.to('cuda', non_blocking=True)
            optimizer.zero_grad()
            outputs = model(x_batch)
            loss = criterion(outputs, y_batch)
            loss.backward()
            optimizer.step()
            train_loss += loss.item()
        
        # 验证步骤
        model.eval()
        val_loss = 0.0
        with torch.no_grad():
            for x_val, y_val in val_loader:
                x_val, y_val = x_val.to('cuda', non_blocking=True), y_val.to('cuda', non_blocking=True)
                outputs = model(x_val)
                val_loss += criterion(outputs, y_val).item()
        
        val_loss /= len(val_loader)
        if val_loss < best_val_loss:
            best_val_loss = val_loss
            counter = 0
        else:
            counter += 1
            if counter >= patience:
                print(f"早停于第{epoch+1}轮")
                break
        model.train()

需在grid_search中拆分训练集为训练+验证集：

from sklearn.model_selection import train_test_split

def grid_search(city, start_year, end_year, param_grid):
    # ... 数据加载与转换代码
    # 拆分训练集为训练和验证
    x_train_split, x_val_split, y_train_split, y_val_split = train_test_split(x_train_tensor, y_train_tensor, test_size=0.2, random_state=42)
    train_dataset = torch.utils.data.TensorDataset(x_train_split, y_train_split)
    val_dataset = torch.utils.data.TensorDataset(x_val_split, y_val_split)
    train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=64, shuffle=True, pin_memory=True)
    val_loader = torch.utils.data.DataLoader(val_dataset, batch_size=64, shuffle=False, pin_memory=True)

    # ... 循环训练时调用修改后的train_model
    train_model(model, criterion, optimizer, train_loader, val_loader, param['epochs'])

3. 增大batch_size
   当前batch_size=32，GPU利用率可能不足，尝试增大到64或128（根据数据量调整），减少每个epoch的迭代次数，提升GPU处理效率。

四、其他小优化 #

• 数据预处理时，用向量操作替代applymap，比如(train_data.iloc[1:, :] > 0).astype(np.float32).values，比applymap更快。
• 在Colab中切换更高规格的GPU（如T4或A100），可通过Runtime > Change runtime type选择。

内容的提问来源于stack exchange，提问作者Lusian