Gensim加载id2word文件失败:dict对象无_load_specials属性
问题分析
你遇到的AttributeError是因为尝试加载的lda_sub_full_35.id2word文件存储的是普通Python字典,而非gensim.corpora.Dictionary对象。corpora.Dictionary.load()方法仅支持加载用该类自身save()方法保存的对象,因此直接调用会触发属性错误。
解决方法
方法一:直接从已加载的LDA模型中提取词典(推荐)
LDA模型加载后自带id2word属性,可直接复用这个映射关系,无需单独加载外部文件:
import numpy as np import os import pandas as pd import gensim from gensim import corpora import datetime import nltk model_name = "lda_sub_full_35" model_for_inference = gensim.models.LdaModel.load(model_name, mmap='r') print('Successfully load the model') # 直接从模型中获取词典映射 lda_dictionary = model_for_inference.id2word # 若模型中的id2word是普通字典,手动转换为Dictionary对象(确保后续doc2bow等功能正常) if isinstance(lda_dictionary, dict): new_dict = corpora.Dictionary() new_dict.id2token = lda_dictionary new_dict.token2id = {token: idx for idx, token in lda_dictionary.items()} lda_dictionary = new_dict
方法二:修复外部id2word文件的加载逻辑
如果必须使用单独保存的lda_sub_full_35.id2word文件,先将其作为普通字典加载,再手动构建corpora.Dictionary对象:
import numpy as np import os import pandas as pd import gensim from gensim import corpora import datetime import nltk import pickle model_name = "lda_sub_full_35" dictionary_name = "lda_sub_full_35.id2word" model_for_inference = gensim.models.LdaModel.load(model_name, mmap='r') print('Successfully load the model') # 加载普通字典文件 with open(dictionary_name, 'rb') as f: id2word_dict = pickle.load(f) # 构建标准Dictionary对象 lda_dictionary = corpora.Dictionary() lda_dictionary.id2token = id2word_dict lda_dictionary.token2id = {token: idx for idx, token in id2word_dict.items()}
后续使用说明
获取到lda_dictionary后,即可用它将新语料转换为词袋向量,再传入LDA模型做主题推断:
# 示例:处理单条新语料 new_text = ["你的", "新语料", "文本"] new_bow = lda_dictionary.doc2bow(new_text) topic_result = model_for_inference[new_bow]
内容的提问来源于stack exchange,提问作者Shiyu
相关产品推荐
相关产品推荐

