使用Selenium和Python爬取Viagogo票价返回空结果求助

常见原因及修复步骤 #

1. 等待动态元素加载 #

Viagogo的票价是动态渲染的，页面加载完成后不会立刻显示所有内容，直接调用find_elements会因为元素未渲染完成返回空列表。需要用显式等待确保元素加载完成：

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

# 最长等待15秒，直到目标元素出现
wait = WebDriverWait(driver, 15)
prices = wait.until(EC.presence_of_all_elements_located((By.XPATH, '//span[@class="t-b fs16"]')))

2. 简化XPATH定位 #

你原有的XPATH层级过深，一旦页面结构微调就会失效。可以直接通过票价元素的特征类定位，更稳定：

# 直接匹配所有带有t-b和fs16类的span元素（对应票价）
prices = driver.find_elements(By.XPATH, '//span[contains(@class, "t-b") and contains(@class, "fs16")]')

3. 规避反爬检测 #

Viagogo会识别自动化工具，需要添加浏览器参数模拟真实用户：

from selenium.webdriver.chrome.options import Options

options = Options()
# 禁用自动化检测标识
options.add_argument("--disable-blink-features=AutomationControlled")
# 设置真实用户代理
options.add_argument("user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36")
driver = webdriver.Chrome(service=s, options=options)

4. 滚动触发内容加载 #

票价列表在页面右侧，可能需要滚动到对应区域才会触发动态加载：

# 滚动到页面底部，触发内容加载
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")

完整修正代码示例 #

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import pandas as pd

# 配置浏览器选项
options = Options()
options.add_argument("--disable-blink-features=AutomationControlled")
options.add_argument("user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36")

s = Service("[]/Downloads/chromedriver/chromedriver.exe")
driver = webdriver.Chrome(service=s, options=options)
driver.get('https://www.viagogo.com/Concert-Tickets/Country-and-Folk/Taylor-Swift-Tickets/E-151214704')

# 滚动到页面底部触发动态加载
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")

# 等待票价元素加载完成
wait = WebDriverWait(driver, 15)
prices = wait.until(EC.presence_of_all_elements_located((By.XPATH, '//span[@class="t-b fs16"]')))

# 提取并打印票价文本
price_list = [price.text for price in prices]
print(price_list)

driver.quit()

内容的提问来源于stack exchange，提问作者MNC