使用Python Selenium无法点击动态网站中的可点击td元素

• 使用Python Selenium的click()命令自动化操作伦敦证交所债券页面时，浏览器打开后目标<td>元素所在列未加载，脚本仅输出"Could not click element"
• 目标：点击Name列下的公司名称（如3i group plc）进入详情页获取工具信息
• 观察到页面中<div _ngcontent-ng-lseg-c170 class>会变为<div _ngcontent-ng-lseg-c170 class="disabled">，怀疑此状态导致元素不可交互

用户原脚本 #

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.chrome.service import Service
from bs4 import BeautifulSoup
import time
url = 'https://www.londonstockexchange.com/live-markets/market-data-dashboard/price-explorer?categories=BONDS'

driver = webdriver.Chrome()
driver.get(url)

wait = WebDriverWait(driver, 10)
table = wait.until(EC.presence_of_element_located((By.ID, 'price-explorer-results-wrapper')))

td_elements = driver.find_elements_by_css_selector('td.clickable.instrument-name.gtm-trackable.td-with-link')

for td in td_elements:
    try:
        WebDriverWait(driver, 10).until(
            EC.element_to_be_clickable((By.CSS_SELECTOR, "td.clickable.instrument-name.gtm-trackable.td-with-link")))
        driver.execute_script("arguments[0].scrollIntoView();", td)
        td.click()
    except:
        print("Could not click element")
driver.quit()

对应HTML片段 #

<table _ngcontent-ng-lseg-c170 id="price-explorer-results-wrapper" class="full-width-price-explorer-results">
  <thead _ngcontent-ng-lseg-c170> ...</thead> == $0
  <tbody _ngcontent-ng-lseg-c170>
      <tr _ngcontent-ng-lseg-c170 class="medium-font-weight slide-panel">
        <td _ngcontent-ng-lseg-c170 class="clickable bold-font-weight instrument-tidm code gtm-trackable td-with-link">...</td>
        <td _ngcontent-ng-lseg-c170 class="clickable instrument-name gtm-trackable td-with-link">
        <span _ngcontent-ng-lseg-c170 class="ellipsed">3i group plc</span></td>
      </tr>
      <tr _ngcontent-ng-lseg-c170 class="medium-font-weight slide-panel"></tr>
      <tr _ngcontent-ng-lseg-c170 class="medium-font-weight slide-panel"></tr>
      
    <div _ngcontent-ng-lseg-c170 class>

1. 核心问题分析 #

页面出现的disabled类说明当前处于不可交互状态，必须等待该状态消失；同时原脚本仅等待表格容器加载，未等待表格数据完全渲染，导致获取的元素列表为空或不可用。

2. 优化后的脚本 #

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time

url = 'https://www.londonstockexchange.com/live-markets/market-data-dashboard/price-explorer?categories=BONDS'

driver = webdriver.Chrome()
driver.get(url)
wait = WebDriverWait(driver, 20)

# 处理Cookie授权弹窗（页面加载后可能出现）
try:
    accept_cookie_btn = wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, 'button#ccc-notify-accept')))
    accept_cookie_btn.click()
except:
    pass

# 等待页面禁用状态消失，确保可交互
wait.until(EC.invisibility_of_element_located((By.CSS_SELECTOR, 'div._ngcontent-ng-lseg-c170.disabled')))

# 等待所有目标td元素加载完成并可点击
td_elements = wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, 'td.clickable.instrument-name.gtm-trackable.td-with-link')))

for index, _ in enumerate(td_elements):
    try:
        # 每次循环重新定位元素，避免页面跳转后出现StaleElement异常
        current_td = wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, f'td.clickable.instrument-name.gtm-trackable.td-with-link:nth-of-type({index+1})')))
        # 滚动元素到视口中央，避免被页面元素遮挡
        driver.execute_script("arguments[0].scrollIntoView({block: 'center'});", current_td)
        current_td.click()
        print(f"成功点击第{index+1}个公司名称")
        
        # 这里可以添加获取详情页信息的代码
        
        # 返回原页面，继续处理下一个元素
        driver.back()
        # 等待原页面重新加载完成
        wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, 'td.clickable.instrument-name.gtm-trackable.td-with-link')))
    except Exception as e:
        print(f"无法点击第{index+1}个元素: {str(e)}")

driver.quit()

3. 关键优化说明 #

• Cookie处理：页面Cookie弹窗会阻止元素交互，需优先处理
• 禁用状态等待：通过invisibility_of_element_located确保页面脱离不可用状态
• 避免Stale元素异常：页面跳转返回后原元素引用失效，每次循环重新定位元素
• 滚动优化：使用scrollIntoView({block: 'center'})确保元素完全显示在视口内
• 延长等待时间：将超时时间从10秒改为20秒，适配页面动态加载的延迟

内容的提问来源于stack exchange，提问作者Yogita Negi