调用Reddit开发者API时遭遇404错误求助
问题解决:Reddit爬虫404错误修复
错误原因
- Subreddit参数格式错误:调用
scrape时传入的是完整Reddit URL(比如https://www.reddit.com/r/funny/)或单个帖子链接,但PRAW的r.subreddit()方法仅接受纯subreddit名称(如funny)。传入完整URL会导致API请求不存在的路径,直接返回404。 - 误用帖子链接作为Subreddit参数:报错里实际传入的是单个帖子链接
https://www.reddit.com/r/augmentedreality/comments/yv7sn8/ar_maximum_distance/,这显然不是subreddit名称,API自然找不到对应资源。
修复方案
1. 修正调用参数
将scrape的第一个参数改为纯subreddit名称,比如:
scrape('funny', 25) # 或针对增强现实板块: scrape('augmentedreality', 25)
2. 完善JSON输出逻辑
原代码中json.dumps(data)仅生成JSON字符串但未做任何处理,建议添加保存到文件或打印的逻辑。
3. (可选)使用PRAW推荐的方法
新版PRAW中get_hot()可替换为更简洁的hot(),功能完全一致。
修正后的完整代码
import praw import json def scrape(subreddit, limit): # 初始化Reddit客户端(替换为你真实的client_id和client_secret) r = praw.Reddit(user_agent='Reddit data organizer 1.0 by /u/reallymemorable', client_id='你的client_id', client_secret='你的client_secret') # 使用hot()替代get_hot(),符合新版PRAW规范 submissions = r.subreddit(subreddit).hot(limit=limit) all_data = [] for submission in submissions: data = {} data['title'] = submission.title data['score'] = submission.score data['url'] = submission.url data['author'] = str(submission.author) if submission.author else None data['subreddit'] = str(submission.subreddit) data['num_comments'] = submission.num_comments data['over_18'] = submission.over_18 data['selftext'] = submission.selftext data['is_self'] = submission.is_self data['name'] = submission.name data['created_utc'] = submission.created_utc data['permalink'] = submission.permalink data['domain'] = submission.domain data['id'] = submission.id data['kind'] = submission.kind all_data.append(data) # 将数据写入JSON文件 with open(f'{subreddit}_posts.json', 'w', encoding='utf-8') as f: json.dump(all_data, f, ensure_ascii=False, indent=4) print(f"数据已保存到{subreddit}_posts.json") # 调用时传入纯subreddit名称 scrape('funny', 25)
额外注意事项
- 确保
client_id和client_secret是从Reddit开发者平台获取的有效凭证,否则可能触发授权错误。 - 新增
submission.author的空值判断,避免帖子作者被删除时抛出异常。
内容的提问来源于stack exchange,提问作者reallymemorable
相关产品推荐
相关产品推荐

