Python Selenium下拉菜单滚动加载全部选项问题求助

我需要获取一个采用滚动加载机制的下拉菜单所有选项，初始仅显示10个，滚动到底部才加载更多。现有代码能打开菜单、获取已加载选项，但滚动操作无法可靠加载全部内容。

现有代码 #

打开下拉菜单 #

open_camera_dropdown = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, 'button.chakra-button.chakra-menu__menu-button.css-1419shu')))
open_camera_dropdown.click()

获取已加载选项 #

camera_options = driver.find_elements(By.CSS_SELECTOR, "div.css-12j7z0z-option, div.css-1wvw4xx-option")
camera_names = [option.find_element(By.CSS_SELECTOR, "p.chakra-text.css-3x00ox").text for option in camera_options]

print("\nAvailable cameras:")
for index, camera_name in enumerate(camera_names, 1):
   print(f"{index}. {camera_name}")

无效的滚动尝试 #

camera_dropdown_element = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CSS_SELECTOR, 'div.css-12j7z0z-option, div.css-1wvw4xx-option')))
    
actions = ActionChains(driver)
actions.move_to_element(camera_dropdown_element)
actions.click_and_hold().move_by_offset(0, 200).release().perform()

解决方案 #

方法1：循环滚动到最后一个选项，直到无新内容加载 #

通过不断定位当前最后一个选项并滚动到它，等待新选项加载，直到选项总数不再变化，确保所有内容都被加载。

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.common.action_chains import ActionChains
import time

# 打开下拉菜单
open_camera_dropdown = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, 'button.chakra-button.chakra-menu__menu-button.css-1419shu')))
open_camera_dropdown.click()

# 等待初始选项加载完成
WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "div.css-12j7z0z-option, div.css-1wvw4xx-option")))

previous_count = 0
current_count = len(driver.find_elements(By.CSS_SELECTOR, "div.css-12j7z0z-option, div.css-1wvw4xx-option"))

# 循环滚动直到选项数量不再增加
while previous_count != current_count:
    previous_count = current_count
    # 获取当前最后一个选项
    last_option = driver.find_elements(By.CSS_SELECTOR, "div.css-12j7z0z-option, div.css-1wvw4xx-option")[-1]
    # 滚动到最后一个选项触发加载
    ActionChains(driver).move_to_element(last_option).perform()
    # 等待新选项加载（可根据页面速度调整时长）
    time.sleep(1)
    # 更新当前选项总数
    current_count = len(driver.find_elements(By.CSS_SELECTOR, "div.css-12j7z0z-option, div.css-1wvw4xx-option"))

# 提取所有相机名称
camera_options = driver.find_elements(By.CSS_SELECTOR, "div.css-12j7z0z-option, div.css-1wvw4xx-option")
camera_names = [option.find_element(By.CSS_SELECTOR, "p.chakra-text.css-3x00ox").text for option in camera_options]

print("\nAvailable cameras:")
for index, camera_name in enumerate(camera_names, 1):
   print(f"{index}. {camera_name}")

方法2：用JavaScript直接滚动下拉菜单容器 #

如果下拉菜单有独立的滚动容器（比如div.chakra-menu__menu），直接用JS控制容器滚动到底部，这种方式更稳定。

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
import time

# 打开下拉菜单
open_camera_dropdown = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, 'button.chakra-button.chakra-menu__menu-button.css-1419shu')))
open_camera_dropdown.click()

# 定位下拉菜单的滚动容器
dropdown_container = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CSS_SELECTOR, 'div.chakra-menu__menu')))

previous_count = 0
current_count = len(driver.find_elements(By.CSS_SELECTOR, "div.css-12j7z0z-option, div.css-1wvw4xx-option"))

while previous_count != current_count:
    previous_count = current_count
    # 执行JS滚动容器到底部
    driver.execute_script("arguments[0].scrollTop = arguments[0].scrollHeight", dropdown_container)
    time.sleep(1)
    current_count = len(driver.find_elements(By.CSS_SELECTOR, "div.css-12j7z0z-option, div.css-1wvw4xx-option"))

# 提取所有相机名称（代码同方法1）
camera_options = driver.find_elements(By.CSS_SELECTOR, "div.css-12j7z0z-option, div.css-1wvw4xx-option")
camera_names = [option.find_element(By.CSS_SELECTOR, "p.chakra-text.css-3x00ox").text for option in camera_options]

print("\nAvailable cameras:")
for index, camera_name in enumerate(camera_names, 1):
   print(f"{index}. {camera_name}")

关键注意事项 #

• 等待时间：time.sleep(1)可根据页面实际加载速度调整，也可以用WebDriverWait等待新元素出现替代固定等待，提升效率。
• 滚动容器定位：如果方法1不生效，优先确认下拉菜单的滚动容器（可通过浏览器开发者工具查看），用JS滚动更可靠。
• 元素定位优化：可以减少重复调用find_elements的次数，比如将元素列表缓存，避免多次DOM查询。

内容的提问来源于stack exchange，提问作者Tigran Papyan