Selenium无法通过XPath/CSS选择器定位可点击元素的问题

问题描述 #

尝试点击页面上的“>”按钮切换日期以获取每日数据表格，但Chrome生成的XPath或CSS选择器无法被Selenium识别，询问Chrome完整XPath是否可直接用于Selenium。

原代码 #

import sys
import os
import pandas as pd
from selenium import webdriver
from pyvirtualdisplay import Display
from bs4 import BeautifulSoup
from selenium import webdriver
import chromedriver_binary
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By


def parse_website():
    # Start a virtual display
    display = Display(visible=0, size=(800, 600))
    display.start()

    try:
        # Set Chrome options with the binary location
        chrome_options = webdriver.ChromeOptions()
        chrome_options.binary_location = "/usr/bin/google-chrome"

        # Initialize Chrome driver
        driver = webdriver.Chrome()

        # Open the desired URL
        url = "https://theanalyst.com/na/2023/08/opta-football-predictions/"
        driver.get(url)

        # Wait for the page to load completely (adjust the time as needed)
        # Parse the page source using BeautifulSoup
        predictions = WebDriverWait(driver, 10).until(
            EC.presence_of_all_elements_located(
                (By.CSS_SELECTOR, "iframe[src*=predictions]")
            )
        )
        element = driver.find_element(By.CSS_SELECTOR,"#pos_3.div.fixtures-header.button:nth-child(3)")
        element.click()
    except Exception as e:
        print(f"An error occurred: {e}")

运行错误 #

An error occurred: Message: no such element: Unable to locate element: {"method":"css selector","selector":"#pos_3.div.fixtures-header.button:nth-child(3)"}

编辑补充：问题源于代码中缺少等待逻辑，感谢Yaroslavm的帮助。

解答 #

1. Chrome生成的完整XPath可以直接用于Selenium，只需使用By.XPATH作为定位方式，示例：

element = WebDriverWait(driver, 10).until(
    EC.element_to_be_clickable((By.XPATH, "这里填Chrome生成的完整XPath"))
)
element.click()

2. 你的代码定位失败的核心原因：

• CSS选择器语法错误：原选择器#pos_3.div.fixtures-header.button:nth-child(3)层级分隔错误，正确写法应为#pos_3 .fixtures-header button:nth-child(3)（ID、类、元素之间用空格分隔，表示后代关系）。
• 未切换到iframe：目标按钮位于页面的iframe内，Selenium默认在主文档中查找元素，必须先切换到对应iframe才能定位内部元素。
• 缺少目标元素的显式等待：仅等待iframe加载完成不够，还需等待按钮元素可点击后再执行点击操作。

修正后的代码示例 #

import sys
import os
import pandas as pd
from selenium import webdriver
from pyvirtualdisplay import Display
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By


def parse_website():
    display = Display(visible=0, size=(800, 600))
    display.start()

    try:
        chrome_options = webdriver.ChromeOptions()
        chrome_options.binary_location = "/usr/bin/google-chrome"
        driver = webdriver.Chrome(options=chrome_options)
        url = "https://theanalyst.com/na/2023/08/opta-football-predictions/"
        driver.get(url)

        # 等待iframe加载并切换到iframe
        iframe = WebDriverWait(driver, 10).until(
            EC.presence_of_element_located((By.CSS_SELECTOR, "iframe[src*=predictions]"))
        )
        driver.switch_to.frame(iframe)

        # 等待目标按钮可点击并执行点击
        next_btn = WebDriverWait(driver, 10).until(
            EC.element_to_be_clickable((By.CSS_SELECTOR, "#pos_3 .fixtures-header button:nth-child(3)"))
        )
        next_btn.click()

    except Exception as e:
        print(f"An error occurred: {e}")
    finally:
        # 资源清理
        driver.quit()
        display.stop()

内容的提问来源于stack exchange，提问作者Michael WS