Selenium爬取网页元素失败：无法获取目标价格数据

目标 #

从目标页面的HTML中提取价格数据 31 954 kr

目标HTML结构 #

<article class="c-product-tile h-full relative product-}">
            <a class="c-product-tile__link" tabindex="0" title="Balance  Spisebord" ng-click="$ctrl.handleClick($event)" pdp-link="c/03-102-01_10991348" outlet="$ctrl.product.type === 'outletProduct'" target="_self" href="/nb-no/mot-oss/butikker/online-outlet/produkt/c/03-102-01_10991348/">
        </a>
    
    
        <div class="flex h-full flex-col relative">
            <div class="c-product-tile__image bg-wild-sand px-6 py-28 mb-4 relative md:px-14 md:py-32 lg:px-12 lg:py-60">
                    <!----><div class="c-aspect-ratio-16x9 anim-fade-in  scale-" ng-if="::$ctrl.product.type !== 'legs' || !$ctrl.product.useLegImages" style="">
            <tile-image base-src="/products/03-102-01_10991348.webp" template="tileImageProductTile" is-product-image="true" move-pressmode-btn=".product- .c-product-tile__image" cachebust="0" mode="pad" aspect-ratio="16x9" default-on-error-url="/layout/images/produkt-default-medium-1.png" bgcolor="transparent" description="Balance  Spisebord"><!----><div class="c-tile-image" ng-if="$ctrl.readyForLoad" style="">
        <!---->
    
        <!---->
        <img src="/layout/images/7x3.gif" lazy-src="https://images.bolia.com/cdn-cgi/image/background=transparent,fit=pad,width=500,format=auto,height=281,quality=81/products/03-102-01_10991348.webp?v=0" lazy-srcset="https://images.bolia.com/cdn-cgi/image/background=transparent,fit=pad,width=340,format=auto,height=191,quality=81/products/03-102-01_10991348.webp?v=0 340w,https://images.bolia.com/cdn-cgi/image/background=transparent,fit=pad,width=540,format=auto,height=303,quality=81/products/03-102-01_10991348.webp?v=0 540w" load-immediately="$ctrl.loadImmediately" sizes="153px" alt="Balance  Spisebord" on-error="/layout/images/produkt-default-medium-1.png" ng-class="{'tile-image__takeover' : $ctrl.takeOver, '': $ctrl.cssClass !== undefined}" srcset="https://images.bolia.com/cdn-cgi/image/background=transparent,fit=pad,width=340,format=auto,height=191,quality=81/products/03-102-01_10991348.webp?v=0 340w,https://images.bolia.com/cdn-cgi/image/background=transparent,fit=pad,width=540,format=auto,height=303,quality=81/products/03-102-01_10991348.webp?v=0 540w">
    </div><!---->
    </tile-image>
        </div><!---->
    
                    <div class="absolute pin my-6">
            <div class="absolute pin-t pin-l">
                <!---->
                <!---->
            </div>
            <!---->
            <div class="absolute pin-l pin-b pin-r flex justify-between">
                <!---->
                <div class="pl-2 pr-4 w-full flex flex-wrap-reverse flex-col justify-self-end justify-end">
                    <!---->
                    <!----><!----><!---->
                </div>
            </div>
        </div>
    
            </div>
                <!----><div ng-if="!$ctrl.noPrice" class="" style="">
            <!---->
    
            <!----><div ng-if="::$ctrl.product.type === 'outletProduct'" class="mb-4">
                <p class="c-product-tile__title c-text-caption m-0 font-bold">
                    <span class="c-text-caption" ng-bind="$ctrl.product.title">Balance  Spisebord</span>
                </p>
                <p class="c-text-caption truncate m-0 hidden md:block" ng-bind="::$ctrl.product.designInformation">Brun marmor</p>
                <p class="c-text-caption m-0" ng-bind="::$ctrl.product.details">God stand</p>
            </div><!---->
    
            <!----><div ng-if="::$ctrl.showFromPrice() || $ctrl.showSalesPrice()" class="m-0 flex flex-wrap items-baseline">
                <!----><p ng-if="!$ctrl.isMyBoliaBoostDiscount()" class="flex flex-wrap c-text-caption m-0">
                    <!----><span ng-if="$ctrl.showSalesPrice()" class="flex flex-wrap items-baseline">
                        <!----><span ng-if="$ctrl.showListPrice()" class="mr-3 font-bold" ng-class="{'font-bold':  $ctrl.showListPrice()}" ng-bind="::$ctrl.product.salesPrice.amount">31&nbsp;954 kr.</span><!---->
                        <span ng-class="{'line-through': $ctrl.showListPrice()}" class="mr-3 line-through" ng-bind="::$ctrl.product.listPrice.amount">63&nbsp;909 kr.</span>
                        <!----><span ng-if="$ctrl.showDiscountText()" ng-style="::$ctrl.splashLabelService.labelStyle()" class="c-text-caption mr-3 flex items-center my-0 px-2 py-1 bg-brandy" style="background: rgb(225, 221, 212); color: rgb(0, 0, 0);">
                            Spar 50%
                        </span><!---->
                    </span><!---->
                    <!---->
                </p><!---->
                <!---->
                <!---->
            </div><!---->
            <!---->
            <!---->
            <!---->
        </div><!---->
        <!---->
    
        </div>
    </article>

我的Python代码（Selenium） #

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome()

driver.get("wwww.scrapedatafromhere.com")

element = WebDriverWait(driver, 10).until(
    EC.visibility_of_element_located((By.XPATH, "(//article[contains(@class, 'c-product-tile')]//p[contains(@class, 'c-product-tile__title')]/span)[1]"))
)

text = element.text
print("Text:", text)

执行结果 #

无任何输出（无报错）

问题分析与解决 #

1. 元素定位错误 #

你当前的XPath定位的是商品标题（即<span>里的Balance Spisebord），而非目标价格数据。目标价格31 954 kr位于绑定$ctrl.product.salesPrice.amount的<span>标签中，该标签带有mr-3 font-bold类。

修正后的XPath可以定位到价格元素：

"//article[contains(@class, 'c-product-tile')]//span[ng-bind='::$ctrl.product.salesPrice.amount']"

或者通过类组合定位：

"//article[contains(@class, 'c-product-tile')]//span[@class='mr-3 font-bold']"

2. URL拼写错误 #

你的driver.get()中的URL是wwww.scrapedatafromhere.com，多了一个w，正确格式应为www.scrapedatafromhere.com，这会导致页面无法正确加载，也是无输出的潜在原因。

3. AngularJS渲染延迟 #

页面使用AngularJS动态渲染数据，建议使用text_to_be_present_in_element等待元素文本加载完成，确保数据已渲染：

修正后的完整代码 #

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome()

# 修正URL拼写
driver.get("www.scrapedatafromhere.com")

# 等待价格元素加载并获取文本
try:
    element = WebDriverWait(driver, 15).until(
        EC.text_to_be_present_in_element(
            (By.XPATH, "//article[contains(@class, 'c-product-tile')]//span[ng-bind='::$ctrl.product.salesPrice.amount']"),
            "kr"  # 等待包含价格单位的文本出现
        )
    )
    price_element = driver.find_element(By.XPATH, "//article[contains(@class, 'c-product-tile')]//span[ng-bind='::$ctrl.product.salesPrice.amount']")
    print("价格:", price_element.text)
finally:
    driver.quit()

内容的提问来源于stack exchange，提问作者Lorenzo Castagno