基于Python的Selenium如何完全展开树形结构？

尝试用Selenium完全展开树形结构以爬取所有子节点，但代码仅能展开第一层，重复执行元素查找和点击操作也无法展开所有层级，还抛出ElementNotInteractableException错误：

Traceback (most recent call last):
  File "/Users/hannes/Desktop/pubmed-scrapper/seleniummeshtree.py", line 18, in <module>
    plus.click()
  File "/Users/hannes/Desktop/pubmed-scrapper/venv/lib/python3.10/site-packages/selenium/webdriver/remote/webelement.py", line 88, in click
    self._execute(Command.CLICK_ELEMENT)
  File "/Users/hannes/Desktop/pubmed-scrapper/venv/lib/python3.10/site-packages/selenium/webdriver/remote/webelement.py", line 396, in _execute
    return self._parent.execute(command, params)
  File "/Users/hannes/Desktop/pubmed-scrapper/venv/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 428, in execute
    self.error_handler.check_response(response)
  File "/Users/hannes/Desktop/pubmed-scrapper/venv/lib/python3.10/site-packages/selenium/webdriver/remote/errorhandler.py", line 243, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable
  (Session info: chrome=106.0.5249.61)
Stacktrace:
0   chromedriver                        0x0000000104935a90 chromedriver + 3889808
1   chromedriver                        0x00000001048c4b54 chromedriver + 3427156
2   chromedriver                        0x00000001045b6108 chromedriver + 221448
3   chromedriver                        0x00000001045e83f0 chromedriver + 426992
4   chromedriver                        0x00000001045de0cc chromedriver + 385228
5   chromedriver                        0x00000001045dda10 chromedriver + 383504
6   chromedriver                        0x0000000104610944 chromedriver + 592196
7   chromedriver                        0x00000001045dc54c chromedriver + 378188
8   chromedriver                        0x00000001049092f8 chromedriver + 3707640
9   chromedriver                        0x000000010490cea8 chromedriver + 3722920
10  chromedriver                        0x00000001049133d8 chromedriver + 3748824
11  chromedriver                        0x000000010490d9a8 chromedriver + 3725736
12  chromedriver                        0x00000001048e8de8 chromedriver + 3575272
13  chromedriver                        0x0000000104927d1c chromedriver + 3833116
14  chromedriver                        0x0000000104927e84 chromedriver + 3833476
15  chromedriver                        0x000000010493c210 chromedriver + 3916304
16  libsystem_pthread.dylib             0x000000019842826c _pthread_start + 148
17  libsystem_pthread.dylib             0x000000019842308c thread_start + 8

当前使用的代码：

from selenium import webdriver
from selenium.webdriver.common.by import By
import time

# load selenium safari webdriver
driver = webdriver.Chrome()
driver.get("https://meshb.nlm.nih.gov/treeView")

# click on the plus-buttons to expand the tree
pluses = driver.find_elements(By.XPATH, "//i[@onclick='openTree(this)']")
for plus in pluses:
    plus.click()
    time.sleep(1)

# click on the plus-buttons to expand the tree
pluses = driver.find_elements(By.XPATH, "//i[@onclick='openTree(this)']")
for plus in pluses:
    plus.click()
    time.sleep(1)

driver.close()

错误原因 #

1. 静态元素列表失效：第一次查找的pluses是固定列表，点击展开后新出现的加号按钮不会被包含，重复两次查找仅能覆盖前两层，无法处理更深层级。
2. 元素不可交互：部分加号按钮未滚动到视图内或未加载完成，直接点击会触发ElementNotInteractableException。
3. 重复点击已展开元素：已点击的加号会变成减号，但原XPATH仍会匹配到这些元素，导致无效点击或报错。

修正后的代码 #

采用循环动态查找元素，直到没有可点击的加号按钮为止，同时处理元素可见性问题：

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import ElementNotInteractableException, StaleElementReferenceException
import time

driver = webdriver.Chrome()
driver.get("https://meshb.nlm.nih.gov/treeView")
driver.maximize_window()

wait = WebDriverWait(driver, 10)

while True:
    # 每次循环重新查找所有未展开的加号按钮
    plus_buttons = wait.until(EC.presence_of_all_elements_located((By.XPATH, "//i[contains(@class, 'fa-plus') and @onclick='openTree(this)']")))
    if not plus_buttons:
        break  # 无更多加号按钮时退出循环
    
    for button in plus_buttons:
        try:
            # 滚动到元素可见位置
            driver.execute_script("arguments[0].scrollIntoView(true);", button)
            time.sleep(0.5)
            button.click()
            time.sleep(0.5)
        except (ElementNotInteractableException, StaleElementReferenceException):
            # 跳过不可交互或已过时的元素
            continue

driver.close()

关键改进点 #

• 动态获取元素：每次循环重新获取加号按钮列表，确保包含新展开层级的按钮。
• 筛选未展开按钮：通过contains(@class, 'fa-plus')仅匹配未展开的加号，避免重复点击已展开元素。
• 处理可见性：用scrollIntoView将元素滚动到视图内，解决不可交互问题。
• 异常捕获：捕获两类常见异常，避免循环中断。

内容的提问来源于stack exchange，提问作者Hannes