Seq2Seq翻译模型训练报错：期望输入形状(4,)却得到(1,)求助

The error ValueError: Error when checking input: expected embedding_1_input to have shape (4,) but got array with shape (1,) happens because your DataGenerator is yielding single samples as 1D arrays, which Keras interprets incorrectly. Here's the breakdown and how to fix it:

Root Cause #

Your model expects input data in the shape (batch_size, sequence_length) (a 2D array), where each entry is a batch of sequences. But your current generator yields individual samples as 1D arrays (e.g., shape (4,) for a sequence of length 4). Keras misinterprets this as a batch of 4 samples, each of length 1—hence the shape mismatch.

Step-by-Step Fixes #

1. Modify the DataGenerator to Yield Batches #

The most efficient fix is to update your generator to produce batches of data instead of single samples. This aligns with how Keras expects input and speeds up training.

Updated DataGenerator code:

def DataGenerator(trainingDataEnglish, trainingDataFrench, batch_size=32):
    while True:
        l = len(trainingDataFrench)
        # Iterate through data in batch-sized chunks
        for i in range(0, l, batch_size):
            # Extract a batch of samples
            batch_x = trainingDataEnglish[i:i+batch_size]
            batch_y = trainingDataFrench[i:i+batch_size]
            yield(batch_x, batch_y)

Adjust your training code to use this batch size and compute the correct number of steps per epoch:

batch_size = 32
steps = len(trainX) // batch_size  # Number of batches per epoch
generator = DataGenerator(trainX, trainY, batch_size=batch_size)
model.fit_generator(generator, epochs=epochs, steps_per_epoch=steps, validation_data=(testX, testY))

2. Ensure Maximum Sequence Lengths Are Correctly Computed #

Your code doesn't show where englishMaxlength and frenchMaxLength are defined. These values must match the maximum length of your sequences after padding. Add this code after creating your tokenizers:

# Calculate maximum sequence lengths from training data
eng_sequences = englishTokenizer.texts_to_sequences(trainEng)
englishMaxlength = max(len(seq) for seq in eng_sequences)

fr_sequences = frenchTokenizer.texts_to_sequences(trainFr)
frenchMaxLength = max(len(seq) for seq in fr_sequences)

3. Verify Vocabulary Sizes Are Properly Defined #

Make sure you've defined frenchVocabsize (used in your model's final Dense layer) just like you did for English:

frenchVocabsize = len(frenchTokenizer.word_index) + 1

Additional Checks #

• Confirm that pad_sequences in encodeSequences uses the correct maxlength parameter (you're already doing this, but double-check it matches englishMaxlength/frenchMaxLength).
• Ensure your training and test data are properly padded to the same sequence lengths.

By implementing these fixes, your model will receive input in the correct shape, and the error should be resolved.

内容的提问来源于stack exchange，提问作者anonymous