寻求适配LSTM的高维少步长多元时间序列降维技术

First off, it's great that you've already validated there's meaningful spatial signal via PCA on single time steps—this gives us a solid starting point for adapting techniques that preserve temporal structure (critical for LSTMs) while taming that 13,000-feature dimensionality, especially with your small 50-sample dataset. Here are the most promising approaches tailored to your use case:

1. Per-Time-Step PCA + Sequential LSTM Input #

Since single-time-step PCA already delivered decent results with an MLP, this is the lowest-hanging fruit to extend to LSTMs:

• For each of your 10 time steps, apply PCA independently to reduce the 13,000 features to a smaller, fixed dimension (e.g., start with 50-100 components, tuned via LOOCV).
• Stack these reduced-dimensional time steps into a sequence tensor ((samples, time_steps, reduced_features)) and feed it directly into an LSTM.
• Why this works: It leverages your existing validated spatial signal while retaining the temporal order LSTMs need. It's computationally lightweight and less prone to overfitting than complex methods with small data.

2. LSTM Autoencoders (Sequence-to-Sequence Dimensionality Reduction) #

If you want to explicitly model temporal dependencies during reduction, an LSTM Autoencoder is a natural fit (and fully compatible with downstream LSTM classifiers):

• Build an encoder LSTM that compresses your 10×13000 sequence into a fixed-length latent vector (or a sequence of latent states).
• Pair it with a decoder LSTM that reconstructs the original sequence—train the autoencoder to minimize reconstruction loss.
• Use the encoder's output (latent vector/states) as the reduced-dimensional input to your LSTM classifier.
• Pro tip for small data: Use heavy regularization (dropout, L2 weight decay) and keep the LSTM layers shallow (1-2 layers, small number of units) to avoid overfitting. You can even fine-tune the encoder with your classification loss after pre-training on reconstruction.

3. Dynamic PCA (Temporal Covariance-Aware Reduction) #

Standard PCA ignores temporal correlations between time steps—Dynamic PCA fixes this by modeling the covariance structure across the entire sequence:

• It computes a shared subspace that captures both spatial feature correlations and temporal dependencies between consecutive time steps.
• The output is a reduced-dimensional sequence where each time step's features are projected into this shared subspace.
• This is a good middle ground between per-time-step PCA and autoencoders: it's linear (so faster to train) but still accounts for temporal structure, which is better than treating each time step in isolation.

4. Temporal Feature Selection #

If you prefer retaining interpretable features (instead of transformed components), try feature selection methods adapted for time series:

• Recursive Feature Elimination (RFE) with LSTM: Train a simple LSTM classifier, iteratively remove the least important features (based on weight magnitudes or gradient contributions), and repeat until you reach a manageable feature count.
• Mutual Information (MI) for Sequences: Compute MI between each feature and the class label, plus MI between each feature and its values in adjacent time steps. Select features that score high on both to retain those with predictive power and temporal relevance.

Practical Recommendations for Your Small Dataset #

• Start with Per-Time-Step PCA + LSTM first—it's fast, builds on your existing results, and minimizes risk of overfitting.
• If you want to capture nonlinear temporal patterns, move to an LSTM Autoencoder with strict regularization.
• Always validate using LOOCV (as you already are) to ensure your results generalize given the small sample size.

内容的提问来源于stack exchange，提问作者Nick