Python Selenium双层循环执行异常：如何修改实现遍历下载？

Let's break down why your code is failing after the first iteration and fix it to reliably traverse all folders and download their files.

Your Original Problem #

Your code runs once successfully, but then throws selenium.common.exceptions.ElementNotInteractableException and the outer loop stops working. The core issue is how you're handling dynamic DOM changes, loop order, and element interactions. Here's your original code for reference:

from selenium.common.exceptions import WebDriverException
from time import sleep

elements = browser.find_elements_by_xpath('//*[@id="dtree0"]/div/a')
names=[]
for elem in elements:
    names.append(elem.text)
print(names)
for id in range (2, 170):
    for i in range(0, len(elements)):
        elements = browser.find_elements_by_xpath ('//*[@id="dtree0"]/div/a')
        elem = elements[i]
        # 仅点击names列表中的元素（当前层级）
        if elem.text in names:
            try:
                elem.click()
            except WebDriverException:
                pass # 忽略不可点击元素
        # browser.find_elements_by_id("stree2").click()
        my_id = "stree{}".format(id)
        browser.find_element_by_id(my_id).click()
        browser.find_element_by_xpath ('/html/body/center[2]/form/table[1]/tbody/tr/td[3]/table/tbody/tr[5]/td[1]/a[1]/img').click ()
        browser.find_element_by_xpath ('/html/body/center[2]/form/table[2]/tbody/tr/td[4]/input').click ()
        browser.find_element_by_xpath ('/html/body/center/form/table[2]/tbody/tr/td[5]/a').click ()
        sleep (5)
        browser.find_element_by_xpath ('//*[@id="personas"]/b').click ()
        browser.find_element_by_xpath('//*[@id="menu_personas"]/a[2]').click()

What's Going Wrong? #

1. Stale Element References: After clicking elements, the page's DOM updates (folders expand, you navigate away and back). The original elements list becomes stale because those elements no longer exist in the current DOM. Even though you re-fetch elements inside the inner loop, the loop order is backwards.
2. Incorrect Loop Order: You're iterating over file IDs first (outer loop) then folders (inner loop). This means you're trying to download all files before fully navigating folders—this doesn't match your goal of opening folders first then downloading their files.
3. Unreliable Sleeps: sleep(5) is a guess at page load time. Sometimes the page isn't ready yet, leading to interactability errors.
4. Lack of Explicit Waits: You're not waiting for elements to be clickable before interacting, which causes the ElementNotInteractableException.

Fixed Code with Explanations #

Here's the revised code that addresses all these issues, plus better error handling:

from selenium.common.exceptions import WebDriverException, NoSuchElementException
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from time import sleep

# Helper function to reuse explicit wait logic (reduces repetition)
def wait_for_clickable(driver, locator_type, locator_value, timeout=10):
    return WebDriverWait(driver, timeout).until(
        EC.element_to_be_clickable((locator_type, locator_value))
    )

# Step 1: Get initial list of folders (only non-empty names)
browser.implicitly_wait(5)  # Basic wait for elements to appear
initial_folders = browser.find_elements(By.XPATH, '//*[@id="dtree0"]/div/a')
folder_names = [folder.text.strip() for folder in initial_folders if folder.text.strip()]
print(f"Found {len(folder_names)} folders to process: {folder_names}")

# Step 2: Iterate over each folder FIRST (correct order)
for folder_name in folder_names:
    try:
        # Re-fetch the folder element every time to avoid stale references
        folder_element = wait_for_clickable(browser, By.XPATH, f'//*[@id="dtree0"]/div/a[text()="{folder_name}"]')
        folder_element.click()
        print(f"✅ Opened folder: {folder_name}")
        
        # Short wait for folder contents to load (adjust based on your page speed)
        sleep(2)

        # Step 3: Now iterate over all file IDs in this folder
        for file_num in range(2, 170):
            file_id = f"stree{file_num}"
            try:
                # Wait for the file element to be ready before clicking
                file_element = wait_for_clickable(browser, By.ID, file_id)
                file_element.click()
                print(f"👉 Clicked file: {file_id}")

                # Perform download steps with explicit waits
                wait_for_clickable(browser, By.XPATH, '/html/body/center[2]/form/table[1]/tbody/tr/td[3]/table/tbody/tr[5]/td[1]/a[1]/img').click()
                wait_for_clickable(browser, By.XPATH, '/html/body/center[2]/form/table[2]/tbody/tr/td[4]/input').click()
                
                # Navigate back to the folder list
                wait_for_clickable(browser, By.XPATH, '/html/body/center/form/table[2]/tbody/tr/td[5]/a').click()
                
                # Wait for the personas menu to load, then navigate back to folder view
                wait_for_clickable(browser, By.XPATH, '//*[@id="personas"]/b').click()
                wait_for_clickable(browser, By.XPATH, '//*[@id="menu_personas"]/a[2]').click()
                
                # Wait for the folder list to reload before next iteration
                wait_for_clickable(browser, By.XPATH, f'//*[@id="dtree0"]/div/a[text()="{folder_name}"]')

            except NoSuchElementException:
                print(f"⚠️ File {file_id} not found in {folder_name}, skipping...")
                # Ensure we're back to the folder view if navigation failed
                try:
                    wait_for_clickable(browser, By.XPATH, '//*[@id="personas"]/b').click()
                    wait_for_clickable(browser, By.XPATH, '//*[@id="menu_personas"]/a[2]').click()
                except:
                    pass
                continue
            except WebDriverException as e:
                print(f"❌ Could not interact with {file_id}: {str(e)}")
                continue

    except NoSuchElementException:
        print(f"⚠️ Folder {folder_name} no longer exists, skipping...")
        continue
    except WebDriverException as e:
        print(f"❌ Could not open folder {folder_name}: {str(e)}")
        continue

Key Improvements #

• Reverse Loop Order: Now we open one folder, download all its files, then move to the next folder—this matches your use case perfectly.
• Explicit Waits: Replaced fixed sleeps with WebDriverWait to ensure elements are actually clickable before interacting, eliminating ElementNotInteractableException.
• Stale Element Fix: We re-fetch folder elements by name every time instead of relying on a stale list from the start.
• Robust Error Handling: Catches specific exceptions (like missing files/folders) and gracefully navigates back to avoid breaking the entire loop.
• Debug Prints: Added clear console messages to track what's happening during execution, which helps with troubleshooting.

Extra Tips for Reliability #

• Avoid Absolute XPaths: The absolute paths like /html/body/center[2]/... are fragile—if the page structure changes slightly, they'll break. Try to use relative XPaths based on element attributes (e.g., //form[@name="downloadForm"]//img[@alt="Download"]).
• Handle Nested Folders: If your folders have subfolders, you'll need to add recursive logic to traverse them. The current code assumes a single level of folders.
• Browser Download Settings: Configure your browser to auto-save files without prompts—this prevents Selenium from getting stuck on download dialogs.
• Implicit vs Explicit Waits: We used a short implicit wait for basic element presence, but explicit waits are better for critical interactions (like clicking download buttons).

内容的提问来源于stack exchange，提问作者user13113347