Python+Selenium点击同class的指定label元素遇阻求助

我用Python和Selenium开发网页爬虫，遇到一个问题：目标页面的fieldset标签下有4个class属性相同的label标签，需要点击其中标注为“Not Sure”的标签，但多次尝试都没成功。

目标页面HTML代码 #

<fieldset>
                            <legend class="sr-only">Are you a?</legend>
                            <input type="hidden" name="_token" value="Nw3CU75ZNiiwSvcXofVUm3hOpcFkvzEc8UdjlPcX">
                            <span id="errMsg"></span>
                            <div aria-live="assertive" class="error-message"></div>
                            <label class="custom-control custom-radio" tabindex="0" for="radio1">
                                <input tabindex="-1" id="radio1" name="radioStep1" type="radio" value='in-network'
                                    class="custom-control-input" aria-describedby="in-network-balloon">
                                <span class="fa-stack" style="vertical-align: top;">
                                    <i class="fal fa-circle fa-stack-2x"></i>
                                    <i style="color:#fff" class="fa fa-circle fa-stack-1x"></i>
                                </span>
                                <span class="custom-control-description" id='in-network-balloon'>
                                    Staying In-Network <span class='sr-only'>If your provider takes your insurance</span>
                                    <i class="far fa-question-circle float-right" data-toggle="tooltip"
                                        data-html="true" data-trigger="focus keypress click" aria-hidden="true"
                                        title="<h3>Staying In-Network</h3>
                                        <p>If your provider takes your insurance</p>"></i>
                                </span>
                                <mark class="inline-cost-div arrow-btn">
                                    <span class="pt-2 border-top border-white">Next
                                        <i class="fa fa-chevron-square-right"></i>
                                    </span>
                                </mark>
                            </label>
                            <label class="custom-control custom-radio" tabindex="0" for="radio2">
                                <input tabindex="-1" id="radio2" name="radioStep1" type="radio" value='out-network'
                                    class="custom-control-input" aria-describedby="out-network-balloon">
                                <span class="fa-stack" style="vertical-align: top;">
                                    <i class="fal fa-circle fa-stack-2x"></i>
                                    <i style="color:#fff" class="fa fa-circle fa-stack-1x"></i>
                                </span>
                                <span class="custom-control-description" id='out-network-balloon'>
                                    Going Out-of-Network <span class='sr-only'>If your provider does not take your insurance.</span>
                                    <i class="far fa-question-circle float-right" data-toggle="tooltip"
                                        data-html="true" data-trigger="focus keypress click" aria-hidden="true"
                                        title="<h3>Going Out-of-Network</h3>
                                        <p>If your provider does not take your insurance.</p>"></i>
                                </span>
                                <mark class="inline-cost-div arrow-btn">
                                    <span class="pt-2 border-top border-white">Next
                                        <i class="fa fa-chevron-square-right"></i>
                                    </span>
                                </mark>
                            </label>
                            <label class="custom-control custom-radio selected" tabindex="0" for="radio3">
                                <input checked tabindex="-1" id="radio3" name="radioStep1" type="radio" value='not-sure'
                                    class="custom-control-input" aria-describedby="not-sure-balloon">
                                <span class="fa-stack" style="vertical-align: top;">
                                    <i class="fal fa-circle fa-stack-2x"></i>
                                    <i style="color:#fff" class="fa fa-circle fa-stack-1x"></i>
                                </span>
                                <span class="custom-control-description" id="not-sure-balloon">
                                    Not Sure <span class='sr-only'>If you don't know if your provider takes your insurance</span>
                                    <i class="far fa-question-circle float-right" data-toggle="tooltip"
                                        data-html="true" data-trigger="focus keypress click" aria-hidden="true"
                                        title="<h3>Not Sure</h3>
                                        <p>If you don't know if your provider takes your insurance</p>"></i>
                                </span>
                                <mark class="inline-cost-div arrow-btn">
                                    <span class="pt-2 border-top border-white">
                                        Next
                                        <i class="fa fa-chevron-square-right"></i>
                                    </span>
                                </mark>
                            </label>
                            <label class="custom-control custom-radio" tabindex="0" for="radio4">
                                <input tabindex="-1" id="radio4" name="radioStep1" type="radio" value='uninsured'
                                    class="custom-control-input" aria-describedby="uninsured-balloon">
                                <span class="fa-stack" style="vertical-align: top;">
                                    <i class="fal fa-circle fa-stack-2x"></i>
                                    <i style="color:#fff" class="fa fa-circle fa-stack-1x"></i>
                                </span>
                                <span class="custom-control-description" id="uninsured-balloon">
                                    Uninsured <span class="sr-only">If you don't have health insurance or if your insurance pays for providers who aren’t in your plan.</span>
                                    <i class="far fa-question-circle float-right" data-toggle="tooltip"
                                        data-html="true" data-trigger="focus keypress click" aria-hidden="true"
                                        title="<h3>Uninsured</h3>
                                        <p>If you don't have health insurance or if your insurance pays for providers who aren’t in your plan.</p>"></i>
                                </span>
                                <mark class="inline-cost-div arrow-btn">
                                    <span class="pt-2 border-top border-white">
                                        Next
                                        <i class="fa fa-chevron-square-right"></i>
                                    </span>
                                </mark>
                            </label>
                        </fieldset>

尝试过的代码 #

label_elements = driver.find_elements_by_css_selector("custom-control custom-radio")
label_elements[1].click()

label_elements = driver.find_elements_by_css_selector("custom-control custom-radio")

for label_element in label_elements:
    label_element.click()

完整代码 #

import selenium
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
import time
import os
from selenium.webdriver.common.action_chains import ActionChains

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

s=Service('/Users/[name]/Desktop/chromedriver')

chromeOptions = Options()
chromeOptions.headless = False
driver = webdriver.Chrome(service=s, options=chromeOptions)


list_data = []

def initalize_browser():
    driver.get("https://www.fairhealthconsumer.org/")
    driver.maximize_window()
   
    driver.find_element(By.XPATH, '//button[@class="mx-auto green-btn btnHref"]').click()
    
    WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@id='radio1' and normalize-space()='Einloggen']"))).click()
    driver.find_element(By.ID, 'not-sure-balloon').click()
    driver.find_elements(By.CLASS_NAME, 'fa-chevron-square-right')[2].click()

    print(driver.page_source)

initalize_browser()


driver.quit

目标页面链接为fairhealthconsumer.org/medical，寻求解决指导。

解决方法 #

1. 修正CSS选择器错误 #

原代码中find_elements_by_css_selector("custom-control custom-radio")是错误的，CSS选择器中类名需要加.，多个类名之间用.连接（表示同时拥有这两个类），正确写法是：

label_elements = driver.find_elements(By.CSS_SELECTOR, ".custom-control.custom-radio")

2. 直接定位“Not Sure”对应的元素 #

方法一：通过radio的value属性定位

因为label关联的radio元素有value='not-sure'，可以直接定位这个radio并点击：

not_sure_radio = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input[value='not-sure']")))
not_sure_radio.click()

方法二：通过label内的文本定位

利用label下的“Not Sure”文本，用XPath定位：

not_sure_label = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//label[contains(.//span[@class='custom-control-description'], 'Not Sure')]")))
not_sure_label.click()

3. 修正页面跳转和等待逻辑 #

原代码中的等待XPath"//button[@id='radio1' and normalize-space()='Einloggen']"完全错误，目标页面是英文，没有“Einloggen”（德语登录）的文本，且radio1是input不是button。同时原代码进入的是首页，需要直接访问目标页面或者正确跳转：

def initalize_browser():
    # 直接访问目标页面
    driver.get("https://www.fairhealthconsumer.org/medical")
    driver.maximize_window()
   
    # 等待并点击Not Sure标签
    not_sure_label = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//label[contains(.//span[@class='custom-control-description'], 'Not Sure')]")))
    not_sure_label.click()
    
    # 点击对应的Next按钮（可以通过label下的mark元素定位）
    next_btn = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//label[contains(.//span[@class='custom-control-description'], 'Not Sure')]//mark[@class='inline-cost-div arrow-btn']")))
    next_btn.click()

    print(driver.page_source)

修改后的完整代码 #

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

s=Service('/Users/[name]/Desktop/chromedriver')

chromeOptions = Options()
chromeOptions.headless = False
driver = webdriver.Chrome(service=s, options=chromeOptions)

list_data = []

def initalize_browser():
    driver.get("https://www.fairhealthconsumer.org/medical")
    driver.maximize_window()
   
    # 定位并点击Not Sure标签
    not_sure_label = WebDriverWait(driver, 20).until(
        EC.element_to_be_clickable((By.XPATH, "//label[contains(.//span[@class='custom-control-description'], 'Not Sure')]"))
    )
    not_sure_label.click()
    
    # 点击对应的Next按钮
    next_btn = WebDriverWait(driver, 20).until(
        EC.element_to_be_clickable((By.XPATH, "//label[contains(.//span[@class='custom-control-description'], 'Not Sure')]//mark[@class='inline-cost-div arrow-btn']"))
    )
    next_btn.click()

    print(driver.page_source)

initalize_browser()

driver.quit()

内容的提问来源于stack exchange，提问作者imherebcsomethingisntworking