TensorFlow中Autoencoder可复现训练失败问题求助

算力阿峰

2026-6-26

解决TensorFlow自动编码器训练无法复现的问题

我在单CPU上基于Autoencoder实现KDDTrain+数据集的异常检测器，已经给TensorFlow设置了随机种子，但每次运行的损失值、网络权重都不一致，两次训练结果如下：

第一次运行：

Epoch 1/5
79/79 [=] - 1s 7ms/step - loss: 0.0779 - val_loss: 0.0628
Epoch 2/5
79/79 [=] - 0s 5ms/step - loss: 0.0514 - val_loss: 0.0397
Epoch 3/5
79/79 [=] - 0s 4ms/step - loss: 0.0311 - val_loss: 0.0236
Epoch 4/5
79/79 [=] - 0s 5ms/step - loss: 0.0193 - val_loss: 0.0157
Epoch 5/5
79/79 [=======================] - 0s 5ms/step - loss: 0.0146 - val_loss: 0.0130

第二次运行：

Epoch 1/5
79/79 [=] - 1s 7ms/step - loss: 0.0726 - val_loss: 0.0589
Epoch 2/5
79/79 [=] - 0s 5ms/step - loss: 0.0475 - val_loss: 0.0363
Epoch 3/5
79/79 [=] - 0s 5ms/step - loss: 0.0297 - val_loss: 0.0233
Epoch 4/5
79/79 [=] - 0s 5ms/step - loss: 0.0185 - val_loss: 0.0144
Epoch 5/5
79/79 [=======================] - 0s 5ms/step - loss: 0.0131 - val_loss: 0.0115

问题根源及修复步骤

1. 特征顺序随机化（最关键问题）

代码中用set筛选数值特征：

numerical_variables = list(set(df.columns.values.tolist()) - set(categorical_variables))

set是无序集合，每次运行时numerical_variables的顺序会随机变化，导致预处理后的数据特征列顺序不一致，模型输入的特征对应关系改变，训练结果自然无法复现。

修复： 用有序列表推导式筛选数值特征，保证顺序固定：

numerical_variables = [col for col in df.columns if col not in categorical_variables and col != 'label']

2. 补全所有随机种子设置

只设置TensorFlow种子不够，还要固定numpy和Python全局随机种子，避免其他库的随机操作影响结果：

import random
random.seed(SEED)
np.random.seed(SEED)
tf.random.set_seed(SEED)
tf.config.experimental.enable_op_determinism()

3. 显式初始化优化器

使用字符串'adam'编译模型时，TensorFlow会隐式创建优化器，可能引入未固定的随机性。显式初始化Adam优化器，确保参数一致：

opt = tf.keras.optimizers.Adam(learning_rate=learning_rate)
autoencoder_model.compile(optimizer=opt, loss=loss_function)

4. 可选：确认数据分割的确定性

虽然原代码的分割逻辑没问题，但用iloc更明确，避免索引异常：

n = int(len(df_preprocessed) * 0.75)
x_train = df_preprocessed.iloc[:n, :]
y_train = integer_labels[:n]

x_test = df_preprocessed.iloc[n:, :]
y_test = integer_labels[n:]

修复后的关键代码片段

import pandas as pd
import numpy as np
import random

import tensorflow as tf
SEED = 42
random.seed(SEED)
np.random.seed(SEED)
tf.random.set_seed(SEED)
tf.config.experimental.enable_op_determinism()

from sklearn.preprocessing import LabelEncoder
from sklearn.preprocessing import MinMaxScaler

def load_and_prepare_data():
    # 加载数据部分不变
    col_names = ["duration", "protocol_type", "service", "flag", "src_bytes", "dst_bytes", "land", "wrong_fragment",
                 "urgent", "hot", "num_failed_logins", "logged_in",
                 "num_compromised", "root_shell", "su_attempted", "num_root", "num_file_creations", "num_shells",
                 "num_access_files", "num_outbound_cmds",
                 "is_host_login", "is_guest_login", "count", "srv_count", "serror_rate", "srv_serror_rate",
                 "rerror_rate", "srv_rerror_rate", "same_srv_rate",
                 "diff_srv_rate", "srv_diff_host_rate", "dst_host_count", "dst_host_srv_count",
                 "dst_host_same_srv_rate", "dst_host_diff_srv_rate",
                 "dst_host_same_src_port_rate", "dst_host_srv_diff_host_rate", "dst_host_serror_rate",
                 "dst_host_srv_serror_rate", "dst_host_rerror_rate",
                 "dst_host_srv_rerror_rate", "label"]

    df = pd.read_csv("../data/KDDTrain+_20Percent.txt", header=None, names=col_names, index_col=False)

    # 替换特征筛选逻辑
    categorical_variables = ['protocol_type', 'service', 'flag', 'land', 'logged_in', 'is_host_login', 'is_guest_login']
    categorical_data = pd.get_dummies(df[categorical_variables])
    
    numerical_variables = [col for col in df.columns if col not in categorical_variables and col != 'label']
    numerical_data = df[numerical_variables].copy()

    df_preprocessed = pd.concat([numerical_data, categorical_data], axis=1)

    # 标签处理不变
    labels = df['label'].copy()
    label_encoder = LabelEncoder()
    integer_labels = label_encoder.fit_transform(labels)

    # 改用iloc分割数据
    n = int(len(df_preprocessed) * 0.75)
    x_train = df_preprocessed.iloc[:n, :]
    y_train = integer_labels[:n]

    x_test = df_preprocessed.iloc[n:, :]
    y_test = integer_labels[n:]

    # 归一化部分不变
    scaler = MinMaxScaler()
    x_train = scaler.fit_transform(x_train)
    x_train = x_train.astype(np.float32)

    x_test = scaler.transform(x_test)
    x_test = x_test.astype(np.float32)

    return x_train, y_train, x_test, y_test

# 模型构建函数不变

def main():
    x_train, y_train, x_test, y_test = load_and_prepare_data()
    input_dim = x_train.shape[1]

    latent_space_dim = 4
    num_neurons_per_layer_list = [16, 48, 64, 96]
    activation_func = 'relu'

    autoencoder_model = build_model(input_dim, latent_space_dim, num_neurons_per_layer_list, activation_func)
    learning_rate = 1e-4
    loss_function = 'mse'
    # 显式创建优化器
    opt = tf.keras.optimizers.Adam(learning_rate=learning_rate)
    autoencoder_model.compile(optimizer=opt, loss=loss_function)

    history = autoencoder_model.fit(x_train, x_train, shuffle=False, epochs=10, batch_size=512,
                                    validation_data=(x_test, x_test))

main()

按照以上修改后，每次运行代码的特征顺序、随机初始化、优化器行为都会完全一致，训练结果就能复现了。

内容的提问来源于stack exchange，提问作者flo