Yelp评论爬取：JSON文件缺失用户名的问题排查与解决

算力阿峰

2026-7-22

问题：Yelp评论爬虫无法获取用户名，JSON输出无用户名字段

你遇到的问题和用户名是否是链接无关——因为你调用的是Yelp的review_feed接口（返回JSON数据），而非直接解析HTML页面。问题出在接口返回的JSON结构中，用户名不在username字段。

原因分析

查看该接口返回的JSON数据，用户信息嵌套在user对象里，用户名对应的字段是name，而非你代码中使用的username。所以r.get('username')会返回None，最终显示为Anonymous。

解决方案

修改获取用户名的代码逻辑，从r['user']['name']中提取：

import requests
from bs4 import BeautifulSoup
import json

url = "https://www.yelp.ie/biz/diwali-indian-restaurant-dublin"
review_url = "https://www.yelp.ie/biz/{biz_id}/review_feed?rl=en&q=&sort_by=relevance_desc&start={start}"
headers = {'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/112.0'}

soup = BeautifulSoup(requests.get(url, headers=headers).content, 'html.parser')
biz_id = soup.select_one('[name=yelp-biz-id]')['content']

reviews = []

for start in range(0, 20, 10):  # <-- increase this range for loading more reviews
    r_url = review_url.format(biz_id=biz_id, start=start)
    data = requests.get(r_url, headers=headers).json()
    for r in data['reviews']:
        review_text = BeautifulSoup(r['comment']['text'], 'html.parser').text
        # 修改这里：从user对象的name字段取用户名
        author = r['user'].get('name', 'Anonymous')
        date = r['localizedDate']
        score = r['rating']
        reviews.append({"author": author, "date": date, "review_text": review_text, "score": score})

with open('reviews.json', 'w') as f:
    json.dump(reviews, f, indent=4)