网络训练过程中损失函数出现异常数值问题求助

Hey there! Let's figure out why your autoencoder training on the NSL-KDD dataset is throwing those weird loss values. I’ll walk through the most likely issues and fixes based on your code snippet.

1. Data Preprocessing is Almost Certainly the Culprit #

NSL-KDD has a mix of categorical and numerical features, and your current loading code doesn’t handle any preprocessing at all. This is the #1 cause of wonky loss values in autoencoders:

• Numerical features like duration or src_bytes have massive value ranges—without scaling, the model’s weights can blow up during training, leading to NaNs or extreme loss numbers.
• Categorical columns (like protocol_type, service, flag) are stored as strings. Feeding raw strings into a neural network will break things immediately, since models can only process numerical data.

2. Your Data Loading Code is Incomplete #

Looking at your load_nsl_kdd_cup_dataset function, it cuts off mid-implementation. Right now, you’re just splitting lines into a list but not:

• Separating feature columns from the final label column
• Converting string values to numerical representations
• Cleaning up malformed or missing rows in the dataset

3. Autoencoder Architecture & Loss Mismatches #

If your autoencoder’s output layer doesn’t align with your input data’s format, you’ll get unexpected loss:

• For example, using sigmoid activation on the output layer when your data isn’t scaled to [0,1], or using mse loss with unnormalized numerical features that have huge variances.

Quick Fixes to Get You Back on Track #

First, Fix Data Preprocessing #

Here’s a cleaned-up, complete version of your data loading function that handles scaling and categorical encoding:

import os
import numpy as np
from sklearn.preprocessing import OneHotEncoder, StandardScaler
from keras.layers import Dense, Input
from keras.models import Model

def load_nsl_kdd_cup_dataset(filename):
    # Define which columns are categorical (indices based on NSL-KDD structure)
    categorical_cols = [1, 2, 3]  # protocol_type, service, flag
    numerical_cols = [i for i in range(41) if i not in categorical_cols]
    
    data_rows = []
    with open(filename, 'r') as f:
        for line in f:
            # Strip whitespace and split by commas
            cleaned_line = line.strip().split(',')
            # Keep first 41 columns as features (last column is the label)
            features = cleaned_line[:41]
            data_rows.append(features)
    
    # Convert to numpy array for processing
    data = np.array(data_rows)
    
    # Scale numerical features to mean=0, std=1
    scaler = StandardScaler()
    numerical_data = scaler.fit_transform(data[:, numerical_cols].astype(float))
    
    # One-hot encode categorical features
    encoder = OneHotEncoder(sparse_output=False)
    categorical_data = encoder.fit_transform(data[:, categorical_cols])
    
    # Combine numerical and categorical features into one array
    processed_features = np.concatenate([numerical_data, categorical_data], axis=1)
    
    return processed_features

Then, Verify Your Autoencoder Setup #

Make sure your input and output shapes match, and your loss function is appropriate for your preprocessed data:

# Load preprocessed data
train_features = load_nsl_kdd_cup_dataset('your_train_file.csv')

# Build the autoencoder
input_dim = train_features.shape[1]
input_layer = Input(shape=(input_dim,))

# Encoder layers
encoder = Dense(64, activation='relu')(input_layer)
encoder = Dense(32, activation='relu')(encoder)

# Decoder layers
decoder = Dense(64, activation='relu')(encoder)
# Use sigmoid if your data is scaled to [0,1], linear if using standardization
output_layer = Dense(input_dim, activation='sigmoid')(decoder)

autoencoder = Model(inputs=input_layer, outputs=output_layer)
# MSE is a solid choice for autoencoder loss with normalized data
autoencoder.compile(optimizer='adam', loss='mse')

Additional Checks #

• Print loss values at every epoch to see if they start normal then diverge (hint: this usually means exploding gradients from bad preprocessing)
• Double-check your dataset for invalid entries (like non-numeric values in numerical columns)

内容的提问来源于stack exchange，提问作者Siamak Parhizkari