Python Selenium循环中Try Except导致耗时过长问题求助

问题描述 #

在Python+Selenium代码中，用Try Except包裹循环查找元素时，单次循环耗时约1分钟。Try块内包含driver.find_element方法，元素未找到时抛出异常，Except块中递减循环计数。希望缩短循环耗时。

原代码 #

def get_element_by_loop(self,elem_name,prefix,suffix,start_no,end_no):
    '''
    此函数根据给定的前缀、起始和结束编号，循环查找指定标签名的元素
    参数1：元素id中的标签名
    参数2：编号前缀
    参数3：循环起始编号
    参数4：循环结束编号
    '''

    retry = end_no
    btn_no = start_no
    while retry>0:
        print(f"retry {retry} button no {btn_no}")
        try:
            elem_no = f"//{elem_name}[@id='{prefix}{btn_no}{suffix}']"
            print(btn_no)
            print(datetime.now())
            if self.checkelembyxpath(elem_no):
                print(elem_no)
                elem = self.driver.find_element(By.XPATH, elem_no)
                retry = 0
                return elem
            else:
                raise Exception("Element not found")
        except:
            btn_no+=1
            retry-=1

运行输出 #

retry 49 button no 11
11
2023-01-06 12:21:53.116641
[29724:3240:0106/122208.282:ERROR:component_installer.cc(153)] Move failed.: The system cannot find the path specified. (0x3)
[29724:3240:0106/122217.878:ERROR:component_installer.cc(153)] Move failed.: The system cannot find the path specified. (0x3)
[29724:3240:0106/122231.926:ERROR:component_installer.cc(153)] Move failed.: The system cannot find the path specified. (0x3)
[29724:3240:0106/122250.540:ERROR:component_installer.cc(153)] Move failed.: The system cannot find the path specified. (0x3)
retry 48 button no 12
12
2023-01-06 12:22:53.187822
[29724:3240:0106/122313.441:ERROR:component_installer.cc(153)] Move failed.: The system cannot find the path specified. (0x3)
[29724:3240:0106/122340.717:ERROR:component_installer.cc(153)] Move failed.: The system cannot find the path specified. (0x3)
retry 47 button no 13
13
2023-01-06 12:23:53.199986
[29724:3240:0106/122412.320:ERROR:component_installer.cc(153)] Move failed.: The system cannot find the path specified. (0x3)
[29724:3240:0106/122448.497:ERROR:component_installer.cc(153)] Move failed.: The system cannot find the path specified. (0x3)
retry 46 button no 14
14
2023-01-06 12:24:53.256596
[29724:3240:0106/122529.520:ERROR:component_installer.cc(153)] Move failed.: The system cannot find the path specified. (0x3)
retry 45 button no 15
15
2023-01-06 12:25:53.305041
[29724:3240:0106/122614.308:ERROR:component_installer.cc(153)] Move failed.: The system cannot find the path specified. (0x3)
retry 44 button no 16
16
2023-01-06 12:26:53.376777

优化方案 #

1. 缩短Selenium等待时间 #

Selenium默认隐式等待时间过长，每次元素查找失败会等待到超时才抛出异常。初始化driver时设置较短的隐式等待：

self.driver.implicitly_wait(5)  # 设为5秒，可根据实际场景调整

如果需要精准控制单个元素的等待，改用显式等待：

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException

# 替换原有的检查+查找逻辑
try:
    elem = WebDriverWait(self.driver, 5).until(
        EC.presence_of_element_located((By.XPATH, elem_no))
    )
    return elem
except TimeoutException:
    btn_no +=1
    retry -=1

2. 避免重复查找元素 #

原代码先调用checkelembyxpath再调用find_element，对同一元素做两次查找，完全多余。直接用find_element或显式等待完成检查+获取操作。

3. 改用更高效的定位方式 #

XPATH定位效率低于ID定位，既然已知元素ID格式，直接用By.ID：

elem_id = f"{prefix}{btn_no}{suffix}"
try:
    elem = self.driver.find_element(By.ID, elem_id)
    return elem
except NoSuchElementException:
    btn_no +=1
    retry -=1

4. 解决Chrome组件错误 #

输出中的component_installer.cc错误是Chrome自动组件更新失败导致的，会拖慢浏览器响应。启动Chrome时禁用组件更新：

from selenium.webdriver.chrome.options import Options

chrome_options = Options()
chrome_options.add_argument("--disable-component-update")
chrome_options.add_argument("--disable-background-networking")
self.driver = webdriver.Chrome(options=chrome_options)

也可以更新Chrome浏览器和ChromeDriver到最新稳定版。

5. 简化循环逻辑 #

原代码retry = end_no的逻辑冗余，直接遍历编号范围更清晰：

def get_element_by_loop(self, elem_name, prefix, suffix, start_no, end_no):
    for btn_no in range(start_no, end_no + 1):
        print(f"Checking button no {btn_no}")
        elem_id = f"{prefix}{btn_no}{suffix}"
        try:
            elem = self.driver.find_element(By.ID, elem_id)
            return elem
        except NoSuchElementException:
            continue
    return None  # 未找到元素时返回None或抛出异常

6. 捕获特定异常 #

原代码用except:捕获所有异常，可能误判其他错误。只捕获Selenium相关的特定异常：

from selenium.common.exceptions import NoSuchElementException, TimeoutException

# 明确指定要捕获的异常类型
except (NoSuchElementException, TimeoutException):
    btn_no +=1
    retry -=1

内容的提问来源于stack exchange，提问作者Punith RJ GS