LSTM二分类模型始终预测1的问题排查与优化咨询

Let's break down your problem step by step—since other models are working fine, the issue is definitely tied to how your LSTM is implemented or how you're feeding data to it.

1. Is there a fundamental mistake in using LSTM for this binary classification task? #

No, LSTMs are fully valid for time-series binary classification. The standard setup (LSTM layers + final Dense sigmoid output + binary crossentropy loss) is correct for this use case. The problem isn't the approach itself—it's in your implementation or data preparation.

2. Are there flaws in your code/model? #

Absolutely, several critical issues stand out:

a. Syntax error in the second LSTM layer #

Your line:

model.add(LSTM(hidden_nodes), activation='hard_sigmoid'))

has misplaced parentheses. The activation parameter belongs inside the LSTM constructor, not outside. It should be:

model.add(LSTM(hidden_nodes, activation='hard_sigmoid'))

This typo might be causing the layer to misconfigure (or even throw an error—maybe you fixed it in your actual code but pasted a mistake here?). Either way, this is a necessary fix.

b. Incorrect time-series input shaping #

You noted each row is a consecutive time step with 55 features, but right now you're reshaping data to (samples, 55, 1)—this treats each of the 55 features as a separate time step with 1 feature. That's backwards!

LSTMs expect input shaped as (number_of_sequences, sequence_length, number_of_features). Since you have 1800 consecutive time steps, you need to create sequential windows (e.g., use the first 10 time steps to predict the 11th's class). For a window size of 10, your input shape would become (1790, 10, 55) (1790 sequences, each 10 time steps long, 55 features per step).

Right now, you're not leveraging any temporal continuity—your LSTM is just processing 55 independent "time steps" per sample, which is no better than a dense network (and likely worse, since LSTMs are built for sequential dependencies).

c. Suboptimal layer choices #

• Using hard_sigmoid as the LSTM activation is unusual. LSTMs typically use tanh for the cell state and standard sigmoid for gates (these are Keras defaults, so you can omit the activation parameter entirely unless you have a specific reason to change it).
• Your dropout rate is only 0.05, which barely provides any regularization. For LSTMs, rates between 0.2–0.5 are more standard.
• You’re using return_sequences=True on the first LSTM but not stacking another sequence-aware layer properly (the syntax error already breaks this). If you want a stacked LSTM, the first layer needs return_sequences=True, but the second should omit it (unless adding more LSTM layers on top).

d. Training oversights #

• No validation data: You’re not monitoring overfitting/underfitting. Adding validation_split=0.1 or passing a validation set to model.fit() will help you see if the model is actually learning or just memorizing noise.
• No early stopping: Your model stops improving after 3 epochs, but you’re forcing it to run 50. Adding EarlyStopping(patience=5, restore_best_weights=True) will halt training when progress stalls and revert to the best weights.
• No data normalization: LSTMs are extremely sensitive to feature scaling. If your 55 features are on wildly different scales, the model will struggle to learn meaningful patterns. Always standardize (Z-score) or normalize (0–1) your features first.

3. What adjustments can you make to get diverse predictions? #

Prioritize fixes in order of impact:

a. Fix input data structure first #

Create sequential windows for your time series with this quick function:

def create_sequences(X, y, window_size):
    sequences = []
    labels = []
    for i in range(len(X) - window_size):
        sequences.append(X[i:i+window_size])
        labels.append(y[i+window_size])
    return np.array(sequences), np.array(labels)

# Example: use 10 past time steps to predict the next step's class
window_size = 10
X_train_seq, y_train_seq = create_sequences(X_train, y_train, window_size)
X_test_seq, y_test_seq = create_sequences(X_test, y_test, window_size)

# Now input shape is (samples, window_size, 55)

Update your LSTM input shape to match:

model.add(LSTM(input_nodes, return_sequences=False, input_shape=(window_size, X_train.shape[1])))

b. Fix model syntax and architecture #

Start simple—use a single LSTM layer before adding complexity:

from sklearn.preprocessing import StandardScaler
from tensorflow.keras import callbacks, optimizers

def train_autoLSTM(X_train, y_train, X_test, y_test, window_size, lstm_units):
    # Normalize data first!
    scaler = StandardScaler()
    X_train_scaled = scaler.fit_transform(X_train.reshape(-1, X_train.shape[-1])).reshape(X_train.shape)
    X_test_scaled = scaler.transform(X_test.reshape(-1, X_test.shape[-1])).reshape(X_test.shape)
    
    model = Sequential()
    model.add(LSTM(lstm_units, input_shape=(window_size, X_train.shape[1])))
    model.add(Dropout(0.3))
    # Explicit zero bias to avoid initial bias towards 1
    model.add(Dense(1, activation="sigmoid", bias_initializer='zeros'))  
    model.compile(optimizer=optimizers.Adam(learning_rate=1e-4), 
                  loss="binary_crossentropy", 
                  metrics=["acc"])
    
    # Add callbacks to optimize training
    callbacks_list = [
        callbacks.EarlyStopping(patience=5, restore_best_weights=True),
        callbacks.ReduceLROnPlateau(factor=0.5, patience=3)
    ]
    
    history = model.fit(X_train_scaled, y_train, epochs=100, 
                        validation_split=0.1, callbacks=callbacks_list)
    
    y_pred = model.predict(X_test_scaled)
    acc = accuracy_score(y_test, y_pred > 0.5)
    return acc, history, y_pred

c. Adjust training hyperparameters #

• Learning rate: Start with 1e-4 (lower than Adam’s default 1e-3) and use ReduceLROnPlateau to decay it if validation loss stalls.
• LSTM units: Start small (32–64) before scaling up to 100+. Your initial calculation for hidden_nodes (using training samples to set units) isn’t standard—stick to powers of 2 for simplicity.
• Bias initialization: If the model still predicts mostly 1s, initialize the final Dense layer’s bias to a negative value (e.g., bias_initializer=keras.initializers.Constant(-2)). This pushes initial outputs towards 0, giving the model room to learn.

d. Debug predictions #

After training, check the distribution of your predictions to identify bias:

print(f"Mean prediction: {np.mean(y_pred):.4f}")
print(f"Prediction min/max: {np.min(y_pred):.4f} / {np.max(y_pred):.4f}")

If the mean is close to 1, the model is biased—this points to input data issues, bias initialization, or a lack of temporal signal. If predictions are spread out but the 0.5 threshold causes all 1s, adjust the threshold (only after fixing core learning issues).

4. Is this dataset unsuitable for LSTM? Should you abandon it? #

Not yet! The fact that other models work means there’s a learnable pattern in the data. The problem right now is that your LSTM isn’t actually using the temporal information (thanks to the input shape mistake).

Once you fix sequential windowing and model syntax, if the LSTM still underperforms, ask: does this task actually require temporal information? If the class label only depends on the current time step’s features (not past steps), LSTMs won’t outperform traditional models. But if there are temporal patterns (e.g., trends over time leading to a class), the LSTM should catch up.

Only abandon LSTM if, after proper setup, it still can’t learn anything meaningful—and even then, try simpler recurrent models like GRUs first (they’re lighter than LSTMs and often work better on small datasets).

内容的提问来源于stack exchange，提问作者LGR