基于Keras实现堆叠卷积+循环层的视频分类器报错求助

Hey there! Let's figure out why you're hitting that IndexError and get your sequence classifier up and running. The issue almost always boils down to input shape mismatches or misusing TimeDistributed with your existing convolution layers. Let's break it down step by step:

1. The Core Problem: Input & Output Shape Misalignment #

Your single-frame model expects a 3D input ((99,99,3)), but when working with sequences (4 frames), you need to feed a 5D tensor to TimeDistributed: (batch_size, timesteps, height, width, channels) — in your case, that's (None, 4, 99, 99, 3).

Additionally, after processing each frame with your convolution stack, you need to make sure the output is a 3D tensor that LSTM can accept: (batch_size, timesteps, features) (here, (None,4,64)). If your TimeDistributed wrapped layers are outputting a 4D tensor (like (None,4,1,64) from leftover spatial dimensions), LSTM will throw an error because it expects 3D input.

2. Correct Implementation Example #

Let's rewrite your architecture properly, building on your existing convolution code:

First, Define Your Reusable Single-Frame Feature Extractor #

This keeps your code clean and reuses the existing convolution logic you've already tested:

from tensorflow.keras import layers, Model

# Single-frame feature extractor (matches your existing conv stack)
input_frame = layers.Input(shape=(99,99,3))
x = layers.Conv2D(32, (3,3), activation='relu')(input_frame)
# Add your other conv layers here...
x = layers.GlobalAveragePooling2D()(x)  # Outputs (None, 64) for single frame
feature_extractor = Model(input_frame, x)

Now Build the Sequence Classifier #

Wrap the feature extractor with TimeDistributed to process each frame in the sequence, then feed the sequence of 64D vectors to LSTM:

# Sequence input: (batch_size, 4 frames, 99,99,3)
sequence_input = layers.Input(shape=(4,99,99,3))

# Apply feature extractor to every frame in the sequence
sequence_features = layers.TimeDistributed(feature_extractor)(sequence_input)
# Output shape here: (None, 4, 64) — perfect for LSTM!

# Add your LSTM layer (note: 1 unit will output a 1D vector, so we need a Dense layer after)
x = layers.LSTM(1)(sequence_features)
# Alternatively, use LSTM(64) for more capacity, then Dense(2)

# Final classification layer (matches your original softmax output)
output = layers.Dense(2, activation='softmax')(x)

sequence_model = Model(sequence_input, output)
sequence_model.summary()

3. Common Mistakes That Cause the Error #

• Feeding separate frames instead of a sequence: If you're passing 4 individual (99,99,3) tensors instead of stacking them into a (4,99,99,3) sequence, TimeDistributed won't have a timestep dimension to iterate over.
• Forgot to wrap GlobalAveragePooling2D in TimeDistributed: If you only wrap the Conv2D layers but not the pooling step, you'll end up with a 4D output (e.g., (None,4,1,64)), which LSTM can't process. Always wrap every layer in your single-frame stack with TimeDistributed.
• Misconfiguring LSTM input: LSTM requires a 3D input ((batch, timesteps, features)). Any extra dimensions (like spatial dimensions from incomplete pooling) will trigger index errors.

4. Quick Check for Shape Issues #

Run model.summary() after building your sequence model. Verify these key shapes:

• Sequence input: (None, 4, 99, 99, 3)
• After TimeDistributed(feature_extractor): (None, 4, 64)
• After LSTM: (None, 1) (if using 1 unit)
• Final output: (None, 2)

If these match, your model should work without the IndexError.

内容的提问来源于stack exchange，提问作者wl2776