使用Selenium无法在目标网站滚动？求可行解决方案

问题分析 #

你的代码在页面加载完成前就执行滚动指令，或者页面的滚动区域并非全局window，导致滚动操作未生效。以下是几种可行的解决方案：

1. 等待页面加载完成后再滚动 #

页面刚打开时可能还在异步加载内容，此时执行滚动会无效。用显式等待等待页面核心元素加载完成后再操作：

import os
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

if __name__ == '__main__':
    os.environ['WDM_LOG'] = '0' 
    options = Options()
    options.add_argument("start-maximized")
    options.add_experimental_option("prefs", {"profile.default_content_setting_values.notifications": 1})    
    options.add_experimental_option("excludeSwitches", ["enable-automation"])
    options.add_experimental_option('excludeSwitches', ['enable-logging'])
    options.add_experimental_option('useAutomationExtension', False)
    options.add_argument('--disable-blink-features=AutomationControlled') 
    srv=Service()
    driver = webdriver.Chrome(service=srv, options=options)    
    
    link = "https://unabated.com/mlb/odds"
    driver.get(link)   
    
    # 等待赔率网格元素加载（可根据页面实际元素调整选择器）
    WebDriverWait(driver, 10).until(
        EC.presence_of_element_located((By.CSS_SELECTOR, "div[class*='OddsGrid']"))
    )
    
    # 滚动到页面底部
    driver.execute_script("window.scrollTo(0, document.body.scrollHeight)")     
    input("Press!") 
    driver.quit()

2. 定位内部滚动容器执行滚动 #

部分网站的内容放在独立的滚动容器中（而非整个浏览器窗口），需要先找到这个容器再滚动：

1. 打开浏览器开发者工具，找到带有overflow-y: auto或overflow: scroll的元素，获取其选择器。
2. 用以下代码替换原滚动逻辑：

# 等待滚动容器加载完成
scroll_container = WebDriverWait(driver, 10).until(
    EC.presence_of_element_located((By.CSS_SELECTOR, "你的滚动容器选择器"))
)

# 滚动容器到底部
driver.execute_script("arguments[0].scrollTop = arguments[0].scrollHeight", scroll_container)

3. 模拟键盘滚动 #

如果前两种方法无效，可模拟键盘按键实现滚动：

import os
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

if __name__ == '__main__':
    os.environ['WDM_LOG'] = '0' 
    options = Options()
    options.add_argument("start-maximized")
    options.add_experimental_option("prefs", {"profile.default_content_setting_values.notifications": 1})    
    options.add_experimental_option("excludeSwitches", ["enable-automation"])
    options.add_experimental_option('excludeSwitches', ['enable-logging'])
    options.add_experimental_option('useAutomationExtension', False)
    options.add_argument('--disable-blink-features=AutomationControlled') 
    srv=Service()
    driver = webdriver.Chrome(service=srv, options=options)    
    
    link = "https://unabated.com/mlb/odds"
    driver.get(link)   
    
    WebDriverWait(driver, 10).until(
        EC.presence_of_element_located((By.CSS_SELECTOR, "div[class*='OddsGrid']"))
    )
    
    # 模拟按5次PageDown键，每次间隔1秒
    action = ActionChains(driver)
    for _ in range(5):
        action.send_keys(Keys.PAGE_DOWN).perform()
        driver.implicitly_wait(1)
    
    input("Press!") 
    driver.quit()

内容的提问来源于stack exchange，提问作者Rapid1898