针对多类别不平衡数据集的随机森林欠采样多数投票方案有效性咨询

Hey there! Great question—your plan to build multiple undersampled datasets, train random forests on each, and use majority voting for final predictions is definitely a viable strategy for tackling your highly imbalanced 7-class dataset. That said, let’s break down why it works, plus some critical factors you might not have considered yet.

Why Your Approach Makes Sense #

• It’s essentially an ensemble of undersampled models, which aligns perfectly with the bagging philosophy behind random forests. By training on different undersampled subsets, you reduce the risk of overfitting to a single skewed data distribution, and majority voting helps smooth out random noise from individual models.
• For your extremely small classes (like the one with only 2,747 samples), undersampling gives these classes a fighting chance—instead of being completely overshadowed by the massive majority classes, models trained on balanced(ish) subsets will pay more attention to minority class patterns.

Key Factors You Might Be Missing #

Let’s dive into the details you’ll want to address to make this approach work well:

• Smart undersampling, not random undersampling
  Randomly cutting down majority classes is easy, but it can throw away critical boundary samples or unique feature distributions that help the model distinguish classes. Instead, try targeted methods like:

  • NearMiss: Keeps majority class samples that are closest to minority class samples, preserving decision boundary information.
  • Tomek Links: Removes overlapping samples between majority and minority classes to reduce confusion.
  • Stratified undersampling with dynamic ratios: Don’t just force all classes to match the smallest class size (that would waste tons of useful data from your large majority classes). Instead, set proportional targets—e.g., reduce the largest classes to 10x the smallest class size, keep mid-sized classes closer to their original counts.

• Account for varying imbalance levels
  Your dataset has a huge range: from 283k+ samples down to 2.7k. If every undersampled subset uses the same ratio, you’ll either lose too much majority class data or still leave some minority classes underrepresented. Mix it up—create some subsets that prioritize preserving majority class details, and others that focus on boosting minority class visibility. This diversity will make your ensemble more robust.

• Ensure model diversity
  Majority voting only works if your individual models don’t all make the same mistakes. Beyond varying undersampling strategies, tweak random forest parameters across subsets: use different max_depth, min_samples_split, or feature subsets for each model. This adds extra diversity to your ensemble.

• Ditch accuracy—use the right evaluation metrics
  Accuracy is useless for imbalanced data (your model could just predict the largest class and get 30%+ accuracy without learning anything). Instead, rely on:

  • Weighted F1-score: Accounts for class imbalance by weighting scores by class size.
  • Confusion matrix: Directly check how well your model performs on each minority class.
  • Recall (per class): Make sure you’re not missing most samples from your smallest classes.
  • Multi-class AUC-ROC: Use One-vs-Rest or One-vs-One to measure overall model performance across classes.

• Compare with sample weight alternatives
  Before committing fully to undersampling, test out using class_weight='balanced' or custom weights in your random forest. This method doesn’t throw away any data (critical for your tiny minority classes) and can sometimes perform just as well—plus it’s simpler and faster to implement. It’s worth running a side-by-side experiment to see which approach works better for your data.

• Watch for overfitting
  Smaller undersampled datasets mean your individual random forests might overfit more easily. Use regularization parameters like max_depth, min_samples_leaf, and max_features to keep models generalizable. Also, use cross-validation on each undersampled subset to ensure your models aren’t just memorizing training data.

• Calculate computational cost
  Training multiple random forests (one per undersampled subset) adds up—especially if you’re working with large original datasets. Make sure your hardware can handle the extra compute time, or start small (e.g., 5-10 subsets instead of 50) to test the approach before scaling up.

Final Thoughts #

Your core idea is solid—ensemble methods are great for imbalanced data, and combining undersampling with random forests plays to both strengths. Just make sure to nail the undersampling strategy, use the right metrics, and test alternatives like sample weights to get the best results.

内容的提问来源于stack exchange，提问作者Kostas Papastamos