基于Surprise的协同过滤推荐系统训练测试集问题求助

1. RMSE测试预测为空的原因 #

你注释的测试集过滤代码逻辑错误：trainset.all_items()返回的是内部item ID（整数类型），而testset里的x[1]是原始item ID（字符串类型），直接用x[1] in trainset.all_items()会把所有测试样本过滤掉，导致testset为空，最终predictions为空，RMSE计算失败。

修复方式：改用训练集的原始item ID字典判断：

# 正确过滤测试集：保留训练集中存在的item
testset = [x for x in testset if x[1] in trainset._raw2inner_id_items]

2. 推荐部分的ValueError问题 #

错误点1：列表误用numpy索引 #

trainset.ur[user_inner_id]返回的是元组列表（每个元素是(内部item ID, 评分)），你用user_ratings[:,0]这种numpy数组索引方式会触发错误。可以用列表推导式提取已评分item ID：

# 提取用户已评分的item内部ID
rated_item_inner_ids = [iid for (iid, _) in trainset.ur[user_inner_id]]

错误点2：未导入defaultdict #

代码中使用defaultdict但未导入，会触发NameError，需在开头添加：

from collections import defaultdict

错误点3：内部ID与原始ID混淆 #

trainset.all_items()返回的是内部item ID，你直接调用to_inner_iid会报错，应该转成原始ID存入推荐列表：

for inner_item_id in trainset.all_items():
    if inner_item_id not in rated_item_inner_ids:
        item_id = trainset.to_raw_iid(inner_item_id)
        items_to_recommend.append(item_id)

修复后的完整代码 #

# import necessary libraries
import numpy as np
from collections import defaultdict
from surprise import SVD
from surprise import Dataset
from surprise import accuracy
from surprise.model_selection import train_test_split

# load the dataset
data = Dataset.load_builtin('ml-100k')

# define the algorithm
algo = SVD()

# split the data into training and testing sets
trainset, testset = train_test_split(data, test_size=.25)

# 正确过滤测试集：保留训练集中存在的item
testset = [x for x in testset if x[1] in trainset._raw2inner_id_items]

# train the algorithm on the training set
algo.fit(trainset)

# make predictions on the testing set
predictions = algo.test(testset)

# calculate RMSE (Root Mean Squared Error)
accuracy.rmse(predictions)

# recommend items to a specific user
user_id = '196'
items_to_recommend = []

# get all the items that the user hasn't rated
if user_id in trainset._raw2inner_id_users:  # check if user id is present in the trainset
    user_inner_id = trainset.to_inner_uid(user_id)
    rated_item_inner_ids = [iid for (iid, _) in trainset.ur[user_inner_id]]
    # 遍历训练集所有item，筛选未评分的
    for inner_item_id in trainset.all_items():
        if inner_item_id not in rated_item_inner_ids:
            item_id = trainset.to_raw_iid(inner_item_id)
            items_to_recommend.append(item_id)

# predict ratings for all items that the user hasn't rated
user_items = [(user_id, item_id, 4.0) for item_id in items_to_recommend] # 4.0 is a placeholder rating
item_ratings = algo.test(user_items)

# if item_ratings is empty, recommend the most popular items
if len(item_ratings) == 0:
    item_popularity = defaultdict(int)
    for (u, i, r) in trainset.all_ratings():
        item_id = trainset.to_raw_iid(i)
        item_popularity[item_id] += 1
    sorted_items = sorted(item_popularity.items(), key=lambda x: x[1], reverse=True)
    items_to_recommend = [iid for (iid, _) in sorted_items][:10]
    print(f"No recommendations found for user {user_id}. Recommending the most popular items.")
else:
    # sort the predicted ratings in descending order
    item_ratings.sort(key=lambda x: x.est, reverse=True)
    # print the top 10 recommended items for the user
    print(f"Top 10 items recommended for user {user_id}:")
    for item in item_ratings[:10]:
        print(f"Item ID: {item.iid}, Estimated Rating: {item.est}")

额外提示 #

• Surprise中trainset的内部ID和原始ID是不同类型，必须用to_inner_uid/to_raw_uid、to_inner_iid/to_raw_iid转换，避免类型不匹配错误。
• 统计热门物品时，trainset.all_ratings()返回的是内部ID，要转成原始ID再统计，否则输出的ID不符合数据集原始格式。

内容的提问来源于stack exchange，提问作者Mark Fisk