如何用Selenium和Python正确等待多个动态下拉菜单加载？

场景与需求 #

爬取IFSC赛事结果页面的联动下拉菜单数据：

• 选择年份：2022（第一个下拉菜单索引1，索引0为最新无赛事年份）
• 选择赛事级别：世界杯/世锦赛（第二个下拉菜单索引1）
• 目标：提取对应的所有Events下拉选项

问题现象 #

前两个下拉菜单的选项能正常提取，但第三个Events下拉菜单几乎总是返回空列表（95%概率）。页面逻辑是：未选League时Events无选项；选择2022后，League下拉自动更新，Events下拉也会随之加载。尝试多种等待方式均无效。

当前代码 #

# Dropdown menus for each choice
year_dd     = self.browser.find_element(By.XPATH, '//select[@id="years"]')
league_dd   = self.browser.find_element(By.XPATH, '//select[@id="indexes"]')
event_dd    = self.browser.find_element(By.XPATH, '//select[@id="events"]')
category_dd = self.browser.find_element(By.XPATH, '//select[@id="categories"]')
            
# Selenium Select class gets objects in dropdown and puts them in corresponding list
years_ob      = Select(year_dd).select_by_index(1)     #0 is most recent year (no events have happened yet)
leagues_ob    = Select(league_dd).select_by_index(1)   #starts at index 1 (World Cups)

wait = WebDriverWait(self.browser, 5)
wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, "option[value='/api/v1/events/*']")))

# I have tried all of the following with similar results
# wait.until(EC.presence_of_all_elements_located((By.XPATH, "//select[@id='events']")))
# wait.until(EC.presence_of_element_located((By.XPATH, "//select[@id='events']"), (By.TAG_NAME, "option")))
# wait.until(EC.element_to_be_clickable((By.TAG_NAME, "option")))
# wait.until(EC.visibility_of_any_elements_located((By.XPATH, "//select[@id='events']")))
# wait.until(lambda x: x.find_element(By.XPATH, "//select[@id='events']/option[text()='IFSC*']"))

year_opts   = Select(year_dd).options
league_opts = Select(league_dd).options
event_opts  = Select(event_dd).options
cat_opts    = Select(category_dd).options

# Extracts the text from the objects and adds to list
years      = [x.text for x in year_opts]
leagues    = [x.text for x in league_opts[1:]]           
events     = [x.text for x in event_opts[1:]]
categories = [x.text for x in cat_opts[1:]]
            
print(years)
print(leagues)
print(events)
print(categories)

尝试过的无效等待方式 #

• 等待option[value='/api/v1/events/*']元素出现
• 等待//select[@id='events']元素出现
• 等待//select[@id='events']下的option元素出现
• 等待option元素可点击
• 等待//select[@id='events']元素可见
• 自定义等待条件：查找包含特定文本的Events选项

怀疑方向 #

怀疑问题和Events下拉菜单中的disabled属性option标签有关，求可行的解决方向。

内容的提问来源于stack exchange，提问作者n0ah