Selenium爬取欧盟资助招标站返回空列表问题求助

1. 修复ChromeDriver配置问题 #

你代码里导入了webdriver_manager但没实际使用，硬编码的/content/chromedriver.exe路径在多数环境下会出错，改成自动管理驱动更稳妥：

from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

# 自动下载并配置对应版本的ChromeDriver
driver_service = Service(ChromeDriverManager().install())
driver = webdriver.Chrome(service=driver_service)

2. 修正元素定位错误 #

你用的wordsToHighlight类名不符合页面实际结构，招标名称实际在.title-wrapper下的链接元素里。另外页面是动态渲染的，用显式等待比隐式等待更可靠，能确保元素完全加载：

driver.get("https://ec.europa.eu/info/funding-tenders/opportunities/portal/screen/opportunities/topic-search;callCode=null;freeTextSearchKeyword=;matchWholeText=true;typeCodes=1,2;statusCodes=31094501,31094502;programmePeriod=2021%20-%202027;programCcm2Id=43108390;programDivisionCode=null;focusAreaCode=null;destinationGroup=null;missionGroup=null;geographicalZonesCode=null;programmeDivisionProspect=null;startDateLte=null;startDateGte=null;crossCuttingPriorityCode=null;cpvCode=null;performanceOfDelivery=null;sortQuery=sortStatus;orderBy=asc;onlyTenders=false;topicListKey=topicSearchTablePageState")

# 显式等待20秒，直到标题元素加载完成
wait = WebDriverWait(driver, 20)
title_elements = wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, ".title-wrapper a.ng-star-inserted")))

# 提取所有非空的招标标题文本
tender_titles = [title.text.strip() for title in title_elements if title.text.strip()]
print(tender_titles)

3. 处理滚动加载问题 #

如果页面仅加载了部分招标信息，需要滚动页面触发更多内容加载：

# 滚动到页面底部
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
# 等待新内容加载完成
wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, ".title-wrapper a.ng-star-inserted")))
# 重新提取全部标题
title_elements = driver.find_elements(By.CSS_SELECTOR, ".title-wrapper a.ng-star-inserted")
tender_titles = [title.text.strip() for title in title_elements if title.text.strip()]
print(tender_titles)

内容的提问来源于stack exchange，提问作者Furkan