关于Many-to-One RNN的截断反向传播及变长序列训练测试的问询

Hey there! Let’s break down your questions about Many-to-One RNNs, truncated backpropagation, and variable-length sequences—these are super common (and critical) topics when working with recurrent models.

1. How to Apply Truncated Backpropagation (TBPTT) to a Many-to-One RNN #

First, let’s recap the setup: a Many-to-One RNN takes a sequence of inputs (T time steps) and outputs a single prediction (like a classification label) based on the final hidden state of the sequence. TBPTT solves the problem of vanishing/exploding gradients in long sequences by cutting the sequence into smaller chunks, computing gradients only within each chunk, and updating parameters incrementally.

Here’s the step-by-step approach to apply it correctly:

• Split long sequences into fixed-length chunks: Choose a truncation window size (e.g., 20 steps). For a sequence longer than this window, split it into consecutive chunks. For example, a 50-step sequence becomes three chunks: 0-19, 20-39, 40-49.
• Forward propagate with persistent hidden states: For each chunk, start with the hidden state from the end of the previous chunk (initialize to zero for the first chunk). Only compute the final Many-to-One output when you reach the last chunk of the sequence.
• Truncate gradients between chunks: After forward propagating a non-final chunk, detach the hidden state from the computation graph. This prevents gradients from flowing backward beyond the current chunk.
• Compute loss and backpropagate only on the final chunk: Once you process the last chunk, use its final hidden state to generate the prediction, calculate loss, and backpropagate gradients only within this chunk (or the most recent N chunks, depending on your TBPTT variant).

Example Code (PyTorch) #

import torch
import torch.nn as nn

# Define a simple Many-to-One RNN
class ManyToOneRNN(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super().__init__()
        self.rnn = nn.RNN(input_size, hidden_size, batch_first=True)
        self.fc = nn.Linear(hidden_size, output_size)
    
    def forward(self, x, hidden):
        out, hidden = self.rnn(x, hidden)
        # Use the final time step's hidden state for prediction
        return self.fc(out[:, -1, :]), hidden

# Initialize model and training components
model = ManyToOneRNN(input_size=10, hidden_size=20, output_size=2)
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters())

# Simulate a long input sequence (length 50)
seq_len = 50
input_seq = torch.randn(1, seq_len, 10)  # (batch_size, seq_len, input_size)
target = torch.tensor([1])

truncation_window = 20
hidden = None

for i in range(0, seq_len, truncation_window):
    end_idx = min(i + truncation_window, seq_len)
    x_chunk = input_seq[:, i:end_idx, :]
    
    # Initialize hidden state if first chunk
    if hidden is None:
        hidden = torch.zeros(1, 1, 20)  # (num_layers, batch_size, hidden_size)
    
    # Forward pass the chunk
    pred, hidden = model(x_chunk, hidden)
    
    # Only compute loss and backprop on the final chunk
    if end_idx == seq_len:
        loss = criterion(pred, target)
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
    else:
        # Detach hidden state to truncate gradients
        hidden = hidden.detach()

2. Training and Testing with Variable-Length Sequences #

Variable-length sequences are handled using padding (to make all sequences in a batch the same length) and masking (to tell the model to ignore padding tokens). Here’s how to implement this:

Training Steps #

1. Pad sequences in a batch: For each batch, pad shorter sequences with zeros (or a special padding token) to match the length of the longest sequence in the batch.
2. Track actual sequence lengths: Keep a list of the original lengths of each sequence in the batch—this lets you retrieve the real final hidden state (not the padding-based final state) for Many-to-One predictions.
3. Use framework tools for masked processing: Libraries like PyTorch have pack_padded_sequence to skip padding during RNN computation, which saves computation and ensures hidden states are only updated with real sequence data.

Testing Steps #

• For single sequences: No padding is needed—just pass the sequence directly to the model and use its final hidden state for prediction.
• For batches: Use the same padding+masking approach as training to process multiple variable-length sequences efficiently.

Example Code (PyTorch) #

from torch.nn.utils.rnn import pack_padded_sequence, pad_packed_sequence

# Simulate a batch of variable-length sequences
batch_inputs = [
    torch.randn(5, 10),  # Length 5
    torch.randn(3, 10),  # Length 3
    torch.randn(7, 10)   # Length 7
]
batch_lengths = [5, 3, 7]

# Pad sequences to the longest length in the batch
padded_inputs = torch.nn.utils.rnn.pad_sequence(batch_inputs, batch_first=True)

# Sort sequences by length (required for pack_padded_sequence)
sorted_lengths, sorted_idx = torch.sort(torch.tensor(batch_lengths), descending=True)
sorted_inputs = padded_inputs[sorted_idx]

# Pack the padded sequence to ignore padding during RNN processing
packed_inputs = pack_padded_sequence(sorted_inputs, sorted_lengths, batch_first=True)

# Forward pass
hidden = torch.zeros(1, 3, 20)
model.eval()
with torch.no_grad():
    packed_out, hidden = model.rnn(packed_inputs, hidden)
    # Unpack the sequence
    out, _ = pad_packed_sequence(packed_out, batch_first=True)
    # Restore original sequence order
    _, unsorted_idx = torch.sort(sorted_idx)
    out = out[unsorted_idx]
    
    # Get the final valid hidden state for each sequence
    final_hidden_states = []
    for i, length in enumerate(batch_lengths):
        final_hidden = out[i, length-1, :]
        final_hidden_states.append(final_hidden)
    final_hidden_states = torch.stack(final_hidden_states)
    
    # Generate predictions
    predictions = model.fc(final_hidden_states)

内容的提问来源于stack exchange，提问作者Aar Vee