Python中使用Selenium抓取特定信息的技术求助

Hey there! I totally get being new to Selenium and feeling stuck on grabbing specific values and storing them in a list. Let's break this down step by step, using the content from your screenshot as our target.

第一步：定位目标元素 #

First, you need to tell Selenium how to find the elements you want to scrape. From your screenshot, the highlighted content looks like it's inside a <span> element with a specific class (a common pattern for displaying values). Here are the most reliable ways to target these elements:

• By Class Name: Perfect if all target elements share a unique class (like value).
• By XPath: Great for narrowing down elements within a specific parent container (super useful if the class name isn't unique on the page).

第二步：提取值并存入列表 #

Once you've located the elements, you'll loop through them, pull out their content, and add it to a list. Here's a complete Python example using Firefox (since you mentioned using the Firefox Inspector):

完整代码示例 #

# 导入必要的Selenium模块
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

# 初始化Firefox浏览器
driver = webdriver.Firefox()
driver.get("https://your-target-website.com")  # 替换成你要爬取的网站URL

try:
    # 等待目标元素加载完成（避免页面未加载完导致的报错）
    # 这里假设目标元素的class是"value"，请根据你的截图实际情况调整
    wait = WebDriverWait(driver, 10)
    target_elements = wait.until(EC.presence_of_all_elements_located((By.CLASS_NAME, "value")))

    # 初始化空列表来存储抓取到的值
    scraped_values = []

    # 遍历所有目标元素，提取文本并添加到列表
    for element in target_elements:
        # 获取元素文本，并用strip()去除多余的空格或换行符
        value_text = element.text.strip()
        if value_text:  # 可选：过滤掉空值
            scraped_values.append(value_text)

    # 打印结果验证
    print("抓取到的值列表：", scraped_values)

finally:
    # 不管操作成功与否，最后关闭浏览器
    driver.quit()

调整定位方式（如果需要） #

If the class name isn't unique, use an XPath to be more specific. For example, if your target spans are inside a div with class data-card, you'd use:

target_elements = wait.until(EC.presence_of_all_elements_located(
    (By.XPATH, "//div[@class='data-card']/span[@class='value']")
))

关键注意事项 #

• 等待元素加载: Always use explicit waits (like WebDriverWait) instead of time.sleep()—it's way more reliable and efficient for dynamic pages.
• 提取属性而非文本: If your target value is stored in an HTML attribute (like data-value instead of visible text), use element.get_attribute("data-value") instead of element.text.
• 处理动态内容: If the content loads after a user action (like clicking a button or scrolling), make sure to trigger that action first before trying to scrape.

内容的提问来源于stack exchange，提问作者Mason Browne