Flask音乐推荐API响应缓慢及500内部服务器错误排查求助

针对你的Flask音乐推荐系统中/recommendation端点的响应慢和偶发500错误问题，以下是针对性的优化和修复方案：

一、解决响应时间过长问题 #

1. 缓存预处理后的数据集 #

每次请求都重新读取CSV并执行字符串处理、合并等操作是最大性能瓶颈。建议提前预处理数据集并缓存到内存或本地文件：

import pickle
from flask import Flask

app = Flask(__name__)

# 全局变量存储缓存数据
spotify_df = None
complete_feature_set_cache = None

# 首次请求前加载预处理好的数据
@app.before_first_request
def load_preprocessed_data():
    global spotify_df, complete_feature_set_cache
    # 提前运行一次预处理，将结果保存为pickle（仅需执行一次）
    # spotify_df = ... # 执行所有数据清洗、合并逻辑
    # with open('spotify_preprocessed.pkl', 'wb') as f:
    #     pickle.dump(spotify_df, f)
    
    # 启动时加载缓存
    with open('spotify_preprocessed.pkl', 'rb') as f:
        spotify_df = pickle.load(f)
    # 预计算特征集
    float_cols = spotify_df.dtypes[spotify_df.dtypes == 'float64'].index.values
    complete_feature_set_cache = create_feature_set(spotify_df, float_cols=float_cols)

2. 批量调用Spotify API #

原代码中循环调用sp.track(x)会产生大量网络请求，改为批量获取：

async def generate_playlist_recos(df, features, nonplaylist_features):
    non_playlist_df = df[df['id'].isin(nonplaylist_features['id'].values)]
    non_playlist_df['sim'] = cosine_similarity(nonplaylist_features.drop('id', axis=1).values,
                                               features.values.reshape(1, -1))[:, 0]
    non_playlist_df_top_40 = non_playlist_df.sort_values('sim', ascending=False).head(40)
    
    # 批量获取专辑封面URL，替代循环调用
    track_ids = non_playlist_df_top_40['id'].tolist()
    tracks_info = sp.tracks(track_ids)
    url_map = {track['id']: track['album']['images'][1]['url'] for track in tracks_info['tracks']}
    non_playlist_df_top_40['url'] = non_playlist_df_top_40['id'].map(url_map)
    
    return non_playlist_df_top_40

3. 替换低效的pandas循环 #

将generate_playlist_feature中的iterrows循环改为向量化操作：

def generate_playlist_feature(complete_feature_set, playlist_df, weight_factor):
    complete_feature_set_playlist = complete_feature_set[
        complete_feature_set['id'].isin(playlist_df['id'].values)
    ].merge(playlist_df[['id', 'date_added']], on='id', how='inner')
    complete_feature_set_nonplaylist = complete_feature_set[
        ~complete_feature_set['id'].isin(playlist_df['id'].values)
    ]

    playlist_feature_set = complete_feature_set_playlist.sort_values('date_added', ascending=False)
    most_recent_date = playlist_feature_set['date_added'].max()
    
    # 向量化计算月份差，替代iterrows
    playlist_feature_set['months_from_recent'] = ((most_recent_date - playlist_feature_set['date_added']).dt.days // 30).astype(int)
    playlist_feature_set['weight'] = weight_factor ** (-playlist_feature_set['months_from_recent'])
    
    # 批量加权计算，避免逐行操作
    feature_cols = playlist_feature_set.columns.difference(['id', 'date_added', 'months_from_recent', 'weight'])
    playlist_feature_set[feature_cols] = playlist_feature_set[feature_cols].multiply(playlist_feature_set['weight'], axis=0)
    
    return playlist_feature_set[feature_cols].sum(axis=0), complete_feature_set_nonplaylist

4. 移除不必要的异步操作 #

普通Flask不原生支持async路由，异步函数可能导致执行异常或性能损耗，将所有async修饰符移除，改为同步函数。

二、解决偶发500内部服务器错误 #

1. 完善错误日志与捕获 #

当前异常捕获仅返回简单错误信息，无法定位具体问题，添加详细日志：

import logging
import traceback

logging.basicConfig(filename='app_error.log', level=logging.ERROR, format='%(asctime)s - %(levelname)s - %(message)s')

@app.route('/recommendation', methods=['GET'])
def recommendation():
    try:
        # ... 业务逻辑
    except Exception as e:
        # 记录完整错误回溯
        logging.error("Recommendation endpoint failed", exc_info=True)
        # 返回详细错误信息用于调试
        return traceback.format_exc(), 500

2. 处理Spotify API异常 #

Spotify API调用可能因网络、权限或资源不存在抛出异常，单独捕获并处理：

from spotipy.exceptions import SpotifyException

def create_necessary_outputs(playlist_name, id_dic, df):
    try:
        playlist_tracks = pd.DataFrame(sp.playlist(id_dic[playlist_name])['tracks']['items'])
    except SpotifyException as e:
        raise ValueError(f"Spotify API request failed: {e.msg}")
    
    playlist_df = playlist_tracks[
        ['track.artists[0].name', 'track.name', 'track.id', 'track.album.images[1].url', 'added_at']
    ]
    playlist_df.columns = ['artist', 'name', 'id', 'url', 'date_added']
    playlist_df['date_added'] = pd.to_datetime(playlist_df['date_added'])
    playlist_df = playlist_df[playlist_df['id'].isin(df['id'].values)].sort_values('date_added', ascending=False)
    
    # 处理空播放列表情况
    if playlist_df.empty:
        raise ValueError("No valid tracks found in the selected playlist")
    
    return playlist_df

3. 检查空DataFrame操作 #

在关键步骤添加空值检查，避免后续操作抛出异常：

def recommendation():
    try:
        # ... 其他逻辑
        playlist_random = create_necessary_outputs(next(iter(tracks.keys())), tracks, spotify_df)
        if playlist_random.empty:
            return jsonify({"error": "No valid tracks in playlist"}), 400
        # ... 后续逻辑
    except ValueError as e:
        return jsonify({"error": str(e)}), 400

内容的提问来源于stack exchange，提问作者Kenvin