如何在Keras中构建输入先经共享层的可变长度循环神经网络

Hey there! Let's work through your Keras RNN problem together—you’re asking about handling variable-length sequences while applying a shared learnable transformation layer f to each input vector x₁, x₂,..., right? Let’s break this down clearly.

First off, your initial thought about a 2D tensor (number_of_x_inputs, x_dimension) works for a single sample (where number_of_x_inputs is the sequence length n for that sample). But for batch training, you need a way to handle samples with different n values. Keras has two main approaches here:

Option A: Padded Sequences + Masking #

The standard approach is to use a 3D tensor with shape (batch_size, None, x_dimension), where None signals that sequence lengths can vary. Here's how to set it up:

• Pad shorter sequences to match the longest sequence in your batch (using tf.keras.preprocessing.sequence.pad_sequences, usually with 0 as the padding value).
• Add a Masking layer or enable mask_zero=True in your RNN layer to tell the model to ignore padding values—this way, the model treats each sequence as its actual length, not the padded length.

Option B: Ragged Tensors (No Padding Needed) #

If you want to avoid padding entirely, Keras supports ragged tensors (irregular tensors that can hold sequences of different lengths directly). The input shape here is still (None, None, x_dimension), but you specify ragged=True in your input layer. This is cleaner for variable-length data, though not all Keras layers support ragged tensors (luckily, the ones we need here do).

The key here is that Keras layers automatically share their parameters every time you call them—so you only need to define f once, then apply it to every time step in your sequence. The easiest way to do this is with the TimeDistributed wrapper, which applies your layer to each element in the sequence independently (using the same weights every time).

Here’s a complete example using padded sequences:

from tensorflow import keras
from tensorflow.keras import layers

# Define your shared learnable transformation layer f
f_layer = layers.Dense(64, activation='relu')  # All x_i use this same layer's weights

# Input layer: accepts variable-length sequences (x_dimension = 32 in this case)
input_seq = layers.Input(shape=(None, 32))

# Apply f to every x_i in the sequence (shared parameters!)
transformed_seq = layers.TimeDistributed(f_layer)(input_seq)

# Add your RNN layer—enable mask_zero=True to ignore padding
rnn_output = layers.LSTM(128, mask_zero=True)(transformed_seq)

# Build and summarize the model
model = keras.Model(inputs=input_seq, outputs=rnn_output)
model.summary()

And here’s the ragged tensor version, if you prefer no padding:

from tensorflow import keras
from tensorflow.keras import layers
import tensorflow as tf

# Same shared f layer
f_layer = layers.Dense(64, activation='relu')

# Input layer for ragged tensors
input_ragged = layers.Input(shape=(None, 32), ragged=True)

# Apply f to each time step
transformed_ragged = layers.TimeDistributed(f_layer)(input_ragged)

# RNN layer works directly with ragged tensors
rnn_output = layers.LSTM(128)(transformed_ragged)

model_ragged = keras.Model(inputs=input_ragged, outputs=rnn_output)

Your 2D tensor (number_of_x_inputs, x_dimension) is fine for a single sample, but you can’t batch these directly (since each sample would have a different first dimension). Using the padded 3D tensor or ragged tensor approach solves this—both let you work with variable-length sequences in batches while keeping the shared f layer intact.

内容的提问来源于stack exchange，提问作者hugom