网页爬虫无法抓取价格元素，返回值为None求助

问题说明 #

抓取咖啡产品的名称、烘焙度（roast）和价格时，现有代码可成功获取名称和烘焙度，但价格返回None。

原代码 #

URLS = ["https://www.thirdwavecoffeeroasters.com/products/vienna-roast","https://www.thirdwavecoffeeroasters.com/products/baarbara-estate","https://www.thirdwavecoffeeroasters.com/products/el-diablo-blend","https://www.thirdwavecoffeeroasters.com/products/organic-signature-filter-coffee-blend","https://www.thirdwavecoffeeroasters.com/products/moka-pot-express-blend-1","https://www.thirdwavecoffeeroasters.com/products/karadykan-estate","https://www.thirdwavecoffeeroasters.com/products/french-roast","https://www.thirdwavecoffeeroasters.com/products/signature-cold-brew-blend","https://www.thirdwavecoffeeroasters.com/products/bettadakhan-estate","https://www.thirdwavecoffeeroasters.com/products/monsoon-malabar-aa"]

for url in range(0,10):
    req=requests.get(URLS[url])
    soup = bs(req.text,"html.parser")
    coffees = soup.find_all("div",class_="col-md-4 col-sm-12 col-xs-12")
    for coffee in coffees:
        name = coffee.find("div",class_="product-details-main").find("ul",class_="uk-breadcrumb uk-text-uppercase").span.text
        roast = coffee.find("div",class_="uk-flex uk-flex-middle uk-width-1-1 coff_type_main").find("p",class_="coff_type uk-margin-small-left uk-text-uppercase").text.split("|")[0]
        prices = coffee.find("div",class_="uk-width-1-1 uk-first-column")

    print(name,roast,price)

问题分析 #

1. 价格元素定位错误：原代码用div.uk-width-1-1.uk-first-column定位，该容器仅为价格的父容器，未直接指向价格文本所在标签。
2. 变量名不一致：代码定义prices但打印时用price，会引发未定义错误。
3. 冗余循环：每个产品页面仅对应一个咖啡产品，内层for coffee in coffees循环无必要。

修复后的代码 #

import requests
from bs4 import BeautifulSoup as bs

URLS = ["https://www.thirdwavecoffeeroasters.com/products/vienna-roast","https://www.thirdwavecoffeeroasters.com/products/baarbara-estate","https://www.thirdwavecoffeeroasters.com/products/el-diablo-blend","https://www.thirdwavecoffeeroasters.com/products/organic-signature-filter-coffee-blend","https://www.thirdwavecoffeeroasters.com/products/moka-pot-express-blend-1","https://www.thirdwavecoffeeroasters.com/products/karadykan-estate","https://www.thirdwavecoffeeroasters.com/products/french-roast","https://www.thirdwavecoffeeroasters.com/products/signature-cold-brew-blend","https://www.thirdwavecoffeeroasters.com/products/bettadakhan-estate","https://www.thirdwavecoffeeroasters.com/products/monsoon-malabar-aa"]

for url in URLS:
    req = requests.get(url)
    soup = bs(req.text, "html.parser")
    coffee = soup.find("div", class_="col-md-4 col-sm-12 col-xs-12")
    if not coffee:
        continue
    # 获取产品名称
    name = coffee.find("div", class_="product-details-main").find("ul", class_="uk-breadcrumb uk-text-uppercase").span.text
    # 获取烘焙度，去除多余空格
    roast = coffee.find("div", class_="uk-flex uk-flex-middle uk-width-1-1 coff_type_main").find("p", class_="coff_type uk-margin-small-left uk-text-uppercase").text.split("|")[0].strip()
    # 定位价格标签并获取文本
    price = coffee.find("span", class_="price").text.strip()
    
    print(name, roast, price)

修复要点 #

• 修正价格选择器：直接使用span.price定位价格元素，确保获取到有效文本。
• 统一变量名：将prices改为price，避免打印时的未定义错误。
• 简化循环：通过find直接获取单个产品容器，减少冗余遍历。
• 添加strip()处理文本，去除多余空格和换行符。

内容的提问来源于stack exchange，提问作者madiha ashfaq