使用Selenium点击表格指定链接获取联想页面替代件编号的求助

你需要对原有代码做以下调整：

• 修复未定义filename变量、分页按钮XPath定位错误的基础问题，避免脚本运行直接报错
• 新增替代件展开逻辑：存在替代件的行左侧有展开箭头，需要点击后才会加载替代件内容，需新增判断和点击逻辑
• 基于当前行定位替代件元素，避免跨行取错数据，无替代件的条目统一标记为无
• 优化文件写入逻辑，避免循环内频繁IO操作降低效率

修改后完整代码 #

from selenium import webdriver
from selenium.webdriver.common.by import By
from time import sleep
import csv

# 配置参数
filename = "lenovo_parts.csv"
url = "https://datacentersupport.lenovo.com/gb/en/products/storage/fibre-channel-switches/b6505-fc-san-switch/3873/parts/display/compatible"
# 初始化驱动，selenium4无需手动指定chromedriver路径，会自动适配
driver = webdriver.Chrome()
driver.get(url)
sleep(5)

# 获取面包屑
bread1 = driver.find_element(By.XPATH, "//span[@class='prod-catagory-name']").text
bread2 = driver.find_element(By.XPATH, "//span[@class='prod-catagory-name']/a").text

# 初始化csv文件，写入表头
with open(filename, 'w', encoding='utf-8-sig', newline='') as f:
    writer = csv.writer(f)
    writer.writerow(["来源链接", "一级分类", "二级分类", "主部件编号", "替代件编号"])

# 获取总页数
pages = int(driver.find_element(By.XPATH, "//div[@class='page-container']/span[@class='icon-s-right active']/preceding-sibling::span[1]").text)

for page in range(pages):
    # 等待表格加载完成
    sleep(2)
    # 获取所有数据行，过滤掉已经展开的替代件子行
    rows = driver.find_elements(By.XPATH, "//table/tbody/tr[not(contains(@class, 'substitute'))]")
    for row in rows:
        # 获取主部件编号
        main_part = row.find_element(By.XPATH, "./td[2]/div").text.strip()
        substitute_part = "无"
        try:
            # 判断是否有展开按钮
            expand_btn = row.find_element(By.XPATH, "./td[1]//span[contains(@class, 'icon-s-right')]")
            # 点击展开
            expand_btn.click()
            sleep(1)
            # 提取替代件编号，替代件在当前行下的class包含substitute的行中
            substitute_ele = row.find_element(By.XPATH, "./following-sibling::tr[contains(@class, 'substitute')][1]//td[2]/div")
            substitute_part = substitute_ele.text.strip()
        except Exception as e:
            # 无替代件则跳过，保持默认值
            pass
        # 写入数据
        with open(filename, 'a', encoding='utf-8-sig', newline='') as f:
            writer = csv.writer(f)
            writer.writerow([url, bread1, bread2, main_part, substitute_part])
    # 不是最后一页则点击下一页
    if page < pages -1:
        driver.find_element(By.XPATH, "//div[@class='page-container']/span[@class='icon-s-right active']").click()
        sleep(3)

driver.quit()

注意事项 #

• 建议使用Selenium 4.x版本运行代码，无需手动下载配置chromedriver
• 可根据网络情况调整sleep的等待时长，避免页面未加载完成导致元素定位失败
• 输出的csv文件默认使用utf-8-sig编码，用Excel打开不会出现乱码

内容的提问来源于stack exchange，提问作者Reggie18