技术求助：如何从application/ld+json提取ratingValue与reviewCount？

我想从指定的application/ld+json数据里提取ratingValue和reviewCount字段，查了不少技术教程都没找到可行方法，特此求助。

示例application/ld+json数据 #

{
   "@context": "http://schema.org",
   "@graph": [
      {
         "@type": "Product",
         "name": "MERV 8 Replacement for Trion Air Bear 20x20x5 (19.63x20.13x4.88) ",
         "description": "MERV 8 Replacement for Trion Air Bear 20x20x5 (19.63x20.13x4.88)  - FilterBuy.com",
         "productID": 30100,
         "sku": "ABR20x20x5M8",
         "mpn": "ABR20x20x5M8",
         "url": "https://filterbuy.com/brand/trion-air-bear-air-filters/20x20x5-air-bear-20x20/merv-8/",
         "itemCondition": "new",
         "brand": "FilterBuy",
         "image": "https://filterbuy.com/media/pla_images/20x25x5AB/20x25x5AB-m8-(x1).jpg",
         "aggregateRating": {
            "@type": "AggregateRating",
            "ratingValue": 4.79926,
            "reviewCount": 538
         }
      }
   ]
}

我编写的Python代码 #

from bs4 import BeautifulSoup
import bs4
import requests
import json
import re
import numpy as np
import csv

urls = ['https://filterbuy.com/brand/trion-air-bear-air-filters/20x20x5-air-bear-20x20',
        'https://filterbuy.com/brand/trion-air-bear-air-filters/16x25x5-air-bear-1400/?selected_merv=11']
for url in urls:
    response = requests.get(url)
    soup = BeautifulSoup(response.text, "lxml")

mervs = BeautifulSoup(response.text, 'lxml').find_all('strong')
product = BeautifulSoup(response.text, 'lxml').find("h1", class_="text-center")
jsonString = soup.find_all('script', type='application/ld+json')[1].text
json_schema = soup.find_all('script', attrs={'type': 'application/ld+json'})[1]
json_file = json.loads(json_schema.get_text())

for i, cart in enumerate(BeautifulSoup(response.text, 'lxml').find_all('form', class_='cart')):
    for tax in cart.attrs:
        if 'data-price' in tax:
            print(product.text, mervs[i].get_text(), [tax], cart[tax], json_file)

解决方案 #

你的代码已经成功加载了JSON数据，只需针对数据结构提取目标字段即可。核心是定位@graph数组里的Product对象，再从其aggregateRating属性中取值。

优化后的完整代码 #

from bs4 import BeautifulSoup
import requests
import json

urls = ['https://filterbuy.com/brand/trion-air-bear-air-filters/20x20x5-air-bear-20x20',
        'https://filterbuy.com/brand/trion-air-bear-air-filters/16x25x5-air-bear-1400/?selected_merv=11']

for url in urls:
    response = requests.get(url)
    soup = BeautifulSoup(response.text, "lxml")
    
    # 提取产品基础信息
    product = soup.find("h1", class_="text-center")
    mervs = soup.find_all('strong')
    
    # 加载并解析JSON-LD数据
    json_scripts = soup.find_all('script', type='application/ld+json')
    if len(json_scripts) < 2:
        print(f"{url} 未找到目标JSON-LD数据")
        continue
    
    try:
        json_data = json.loads(json_scripts[1].get_text())
    except json.JSONDecodeError:
        print(f"{url} JSON-LD数据解析失败")
        continue
    
    # 提取评分和评论数
    rating_value = None
    review_count = None
    for item in json_data.get('@graph', []):
        if item.get('@type') == 'Product' and 'aggregateRating' in item:
            rating_value = item['aggregateRating'].get('ratingValue')
            review_count = item['aggregateRating'].get('reviewCount')
            break
    
    # 输出整合信息
    for i, cart in enumerate(soup.find_all('form', class_='cart')):
        for attr in cart.attrs:
            if 'data-price' in attr:
                product_name = product.text.strip() if product else '未知产品'
                merv = mervs[i].get_text().strip() if i < len(mervs) else '未知MERV'
                price = cart[attr]
                
                print(f"产品：{product_name}")
                print(f"MERV等级：{merv}")
                print(f"价格：{price}")
                if rating_value and review_count:
                    print(f"评分：{rating_value}，评论数：{review_count}")
                else:
                    print("无评分数据")
                print("---")

关键说明 #

1. 避免重复创建BeautifulSoup对象：原代码多次解析HTML，优化后只创建一次，提升效率
2. 异常处理：加入JSON解析异常和脚本存在性判断，避免页面结构变化导致崩溃
3. 精准定位数据：遍历@graph数组筛选Product类型对象，再从aggregateRating中提取目标字段

内容的提问来源于stack exchange，提问作者Mike