生成维基文本向量时遇list索引错误，求代码排查解决

错误原因 #

你遇到的list indices must be integers or slices, not str错误，是因为把word2idx定义成了列表，但列表只能用整数当索引，你却用字符串（单词）去访问，自然报错。另外代码里还有两个明显问题：

• 遍历parsed的循环写法完全错误，len(range(parsed))是无效语法
• 原getVocab没清理逗号、撇号这类标点，导致词汇里混入带符号的单词，比如interpreted,，影响向量生成准确性

修正后的代码 #

def getVocab(inputString):
    inputString = inputString.lower()
    # 清理所有非字母数字和空格的字符，保留单词本身
    inputString = ''.join([c for c in inputString if c.isalnum() or c == ' '])
    parsed = inputString.split()
    vocab = set(parsed)
    return vocab, parsed 

wikipediaText = 'Python is an interpreted, high-level, general-purpose programming language.   Created by Guido van Rossum and first released in 1991, Python`s design philosophy emphasizes code  readability with its notable use of significant whitespace.'

vocab, _= getVocab(wikipediaText)
print(vocab)


def generateVector(inputString):
    vocab, parsed = getVocab(inputString)
    # 把word2idx改成字典，用来映射单词到索引
    word2idx = {}
    for idx, word in enumerate(vocab):
        word2idx[word] = idx
    
    vector = []
    # 直接遍历parsed里的每个单词，取对应的索引
    for word in parsed:
        vector.append(word2idx[word])

    return vector

print(generateVector(wikipediaText))

关键修改说明 #

• 将word2idx从列表[]改为字典{}，字典支持字符串作为键，完美实现“单词→索引”的映射
• 修正getVocab函数，过滤掉所有非字母数字和空格的字符，确保词汇都是干净的单词
• 把错误的循环for word in len(range(parsed))改成for word in parsed，正确遍历每个单词生成向量

内容的提问来源于stack exchange，提问作者Lina