Keras训练Reddit帖子子版块预测模型时准确率与Loss波动问题

Hey there! The fluctuating accuracy and loss you're seeing typically stem from a mix of data preprocessing quirks, training setup choices, and model configuration tweaks. Let's walk through actionable fixes to stabilize your metrics and boost performance:

1. Fix Data Preprocessing Issues #

Max Sequence Length Miscalculation #

Your current way of calculating max_sentence_len is incorrect—len(df.title.max()) gives the character length of the longest title, not the token count (which is what matters for sequence padding). This leads to wrong padding lengths, which can throw off model training and cause unstable metrics.

Fix it with:

# Calculate max token length after tokenization
title_sequences = title_tokenizer.texts_to_sequences(df.title)
max_sentence_len = max(len(seq) for seq in title_sequences)

Improve Dataset Handling #

• Sufficient Shuffling: A shuffle buffer of 50 is way too small—this means your model is seeing the same tiny subset of data repeatedly, leading to erratic weight updates. Use a buffer size equal to your dataset size (if memory allows) or a large fraction:

  dataset = dataset.shuffle(buffer_size=len(df))  # Full dataset shuffle for proper randomization
  

• Split Train/Validation Sets: Using repeat(None) with fixed steps_per_epoch prevents you from tracking real generalization performance. Split your data to monitor validation metrics and avoid overfitting:

  from sklearn.model_selection import train_test_split
  
  # Split dataframe into 80% train, 20% validation
  train_df, val_df = train_test_split(df, test_size=0.2, random_state=42)
  
  # Helper function to create datasets
  def create_dataset(dataframe):
      dataset = tf.data.Dataset.from_tensor_slices((dataframe.title, dataframe.subreddit))
      dataset = dataset.map(lambda x, y: tf.py_function(preprocess, [x, y], [tf.int64, tf.int64]))
      dataset = dataset.map(set_shape_func)
      dataset = dataset.shuffle(len(dataframe)).batch(128).prefetch(tf.data.AUTOTUNE)
      return dataset
  
  train_dataset = create_dataset(train_df)
  val_dataset = create_dataset(val_df)
  

• Switch to TensorFlow-Native Text Processing: The tf.py_function call introduces overhead and consistency issues. Replace the old Tokenizer with TextVectorization for seamless TensorFlow integration:

  # Initialize vectorizer (replaces Tokenizer)
  title_vectorizer = tf.keras.layers.TextVectorization(
      max_tokens=title_voc_size + 1,
      output_sequence_length=max_sentence_len
  )
  title_vectorizer.adapt(df.title)
  
  # You can now use this layer directly in your model, eliminating manual sequence/padding steps
  

2. Adjust Model & Training Configuration #

Optimize the Optimizer #

SGD with a fixed learning rate of 0.015 is prone to large fluctuations because it lacks adaptive learning or momentum. Switch to Adam (which handles learning rates automatically) or add momentum to SGD:

Option 1: Use Adam

model.compile(keras.optimizers.Adam(learning_rate=lr), 'categorical_crossentropy', metrics=['acc'])

Option 2: SGD with Momentum

model.compile(keras.optimizers.SGD(lr, momentum=0.9), 'categorical_crossentropy', metrics=['acc'])

Simplify Model to Reduce Overfitting #

Your current model has two dense layers with 256 units each after LSTMs, which might be overparameterized for your dataset. Try these tweaks:

• Replace the second LSTM with GlobalAveragePooling1D to reduce parameter count:

  inputs = keras.Input(max_sentence_len)
  x = keras.layers.Embedding(title_voc_size+1, 100)(inputs)
  x = keras.layers.Bidirectional(keras.layers.LSTM(256, return_sequences=True))(x)
  x = keras.layers.GlobalAveragePooling1D()(x)  # Replaces the second LSTM, cuts down parameters
  x = keras.layers.Dropout(0.3)(x)
  x = keras.layers.Dense(128, activation='relu')(x)  # Reduce dense layer size
  outputs = keras.layers.Dense(subreddits_size + 1, activation='softmax')(x)
  

• Use pre-trained word embeddings (like GloVe) to give your model better starting features:

  # Load GloVe embeddings (adjust path to your file)
  glove_path = 'glove.6B.100d.txt'
  embedding_index = {}
  with open(glove_path) as f:
      for line in f:
          word, coefs = line.split(maxsplit=1)
          coefs = np.fromstring(coefs, 'f', sep=' ')
          embedding_index[word] = coefs
  
  # Create embedding matrix for your vocabulary
  embedding_matrix = np.zeros((title_voc_size + 1, 100))
  for word, i in title_tokenizer.word_index.items():
      embedding_vector = embedding_index.get(word)
      if embedding_vector is not None:
          embedding_matrix[i] = embedding_vector
  
  # Use pre-trained embeddings in the model (set trainable=True if you want to fine-tune)
  x = keras.layers.Embedding(title_voc_size+1, 100, weights=[embedding_matrix], trainable=False)(inputs)
  

Add Training Stabilizers #

• Early Stopping: Stop training when validation loss stops improving to avoid overfitting and unnecessary fluctuations:

  early_stopping = keras.callbacks.EarlyStopping(monitor='val_loss', patience=5, restore_best_weights=True)
  

• Learning Rate Scheduling: Reduce the learning rate when validation metrics plateau to fine-tune smoothly:

  lr_scheduler = keras.callbacks.ReduceLROnPlateau(monitor='val_loss', factor=0.5, patience=3, min_lr=1e-5)
  

• Update Training Loop: Use the train/validation datasets and add callbacks:

  history = model.fit(
      train_dataset,
      epochs=500,
      validation_data=val_dataset,
      callbacks=[early_stopping, lr_scheduler]
  )
  

Fix Evaluation #

You're evaluating on the infinite-repeating training dataset, which gives misleading results. Evaluate on the validation set instead:

loss, acc = model.evaluate(val_dataset)

3. Additional Tips #

• Check Class Balance: If your subreddit classes are imbalanced, metrics will fluctuate. Use class_weight in model.fit or oversample/undersample minority classes.
• Batch Size Adjustment: If 128 is too large for your memory, try 64—but ensure the size is consistent. Smaller batches can cause more fluctuation, so pair with proper shuffling.

内容的提问来源于stack exchange，提问作者Tomergt45