PyTorch深度学习聊天机器人遇IndexError:列表索引越界错误求助

Let's break down your error and fix it step by step:

What's Causing the Error? #

The IndexError on line 198 happens because some of your "question-answer pairs" aren't actually pairs at all. The code assumes every entry in pairs is a 2-element list (input sentence + response sentence), but some entries have only 0 or 1 elements. When filterPair tries to access p[0] or p[1] on these invalid entries, it throws the "index out of range" error.

This usually happens because:

• Some lines in your formatted_movie_lines.txt file don't have a tab (\t) separator (e.g., empty lines, corrupted lines from the data formatting step).
• The readVocs function blindly splits every line by \t without checking if the split produces exactly two elements.

How to Fix It #

You can resolve this with one of two straightforward fixes:

Fix 1: Filter Invalid Pairs When Loading Data #

Update your readVocs function to only keep lines that split into valid 2-element pairs:

def readVocs(datafile, corpus_name):
    print("Reading lines...")
    # Read the file and split into lines
    lines = open(datafile, encoding='utf-8').read().strip().split('\n')
    # Split lines into pairs, normalize, and filter out invalid entries
    pairs = []
    for l in lines:
        split_line = l.split('\t')
        # Only keep lines that split into exactly two parts
        if len(split_line) == 2:
            pairs.append([normalizeString(split_line[0]), normalizeString(split_line[1])])
    voc = Voc(corpus_name)
    return voc, pairs

Fix 2: Add Validation in the Filter Function #

Modify filterPair to first check if the pair is valid before checking sentence lengths:

def filterPair(p):
    # First ensure we're dealing with a valid 2-element pair
    if len(p) != 2:
        return False
    # Input sequences need to preserve the last word for EOS token
    return len(p[0].split(' ')) < MAX_LENGTH and len(p[1].split(' ')) < MAX_LENGTH

Bonus: Prevent Invalid Lines in the First Place #

To avoid this issue from recurring, you can also add a check in the extractSentencePairs function to ensure only valid pairs are written to formatted_movie_lines.txt:

def extractSentencePairs(conversations):
    qa_pairs = []
    for conversation in conversations:
        # Iterate over all the lines of the conversation
        for i in range(len(conversation["lines"]) - 1):
            inputLine = conversation["lines"][i]["text"].strip()
            targetLine = conversation["lines"][i+1]["text"].strip()
            # Filter wrong samples (if one of the lists is empty OR contains invalid characters)
            if inputLine and targetLine and "\t" not in inputLine and "\t" not in targetLine:
                qa_pairs.append([inputLine, targetLine])
    return qa_pairs

This ensures neither the input nor response contains a tab character (which would break the file formatting later).

内容的提问来源于stack exchange，提问作者Timothy