Python Selenium获取视频链接异常：仅返回单个链接问题求助

我尝试解析并展示爬取到的视频链接列表，但运行代码后仅输出1个链接。变量linkvideo2仅获取到第一个视频链接，怀疑是否无法在循环中使用CSS_SELECTOR？

原代码 #

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.support import expected_conditions as EC

import time 

name = 'hello world'
profile_path = r"C:/Users/Федорчик/Desktop"
options=Options()
options.set_preference('profile', profile_path)
driver = webdriver.Firefox(options=options)
#driver = webdriver.Firefox(r"C:/Users/Федорчик/Desktop")
driver.get('https://www.youtube.com')
id_serth = driver.find_element(By.NAME, "search_query")
id_serth.send_keys(name)
button_serth = driver.find_element(By.ID, "search-icon-legacy")
time.sleep(4)
button_serth.click()
time.sleep(4)
button_filtr = driver.find_element(By.CLASS_NAME ,"ytd-toggle-button-renderer")
button_filtr.click()
time.sleep(4)
button_filtrtode=driver.find_element(By.CLASS_NAME, "ytd-search-filter-renderer")
button_filtrtode.click()
time.sleep(4)

urltek = driver.current_url


linkvideo2 = driver.find_elements(By.CSS_SELECTOR, 'ytd-video-renderer.style-scope:nth-child(1) > div:nth-child(1) > div:nth-child(2) > div:nth-child(1) > div:nth-child(1) > h3:nth-child(1) > a:nth-child(2)')
links=[]
for i in linkvideo2:
    links.append(i.get_attribute('href'))
print(len(links))

print (urltek)

运行结果 #

1
https://www.youtube.com/results?search_query=hello+world&sp=EgIIAQ%253D%253D

解决方案 #

问题根源是你的CSS选择器里的:nth-child(1)限定了只匹配第一个ytd-video-renderer元素，导致find_elements只能返回1个结果，和循环无关。

1. 简化并修正CSS选择器

去掉选择器中的:nth-child(1)限制，直接定位所有视频条目里的标题链接，推荐使用更简洁的选择器：ytd-video-renderer h3 a，它能匹配所有视频的标题跳转链接。

2. 替换time.sleep为显式等待

用WebDriverWait等待元素加载完成，避免因页面未完全加载导致漏抓元素，稳定性更高。

修改后的关键代码片段

# 等待搜索结果中的视频元素加载完成
WebDriverWait(driver, 10).until(
    EC.presence_of_all_elements_located((By.CSS_SELECTOR, 'ytd-video-renderer h3 a'))
)

# 获取所有视频链接
linkvideo2 = driver.find_elements(By.CSS_SELECTOR, 'ytd-video-renderer h3 a')
links = [i.get_attribute('href') for i in linkvideo2]
print(len(links))

完整优化后的代码

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.support import expected_conditions as EC

name = 'hello world'
profile_path = r"C:/Users/Федорчик/Desktop"
options=Options()
options.set_preference('profile', profile_path)
driver = webdriver.Firefox(options=options)
driver.get('https://www.youtube.com')

# 搜索操作
search_box = WebDriverWait(driver, 10).until(
    EC.presence_of_element_located((By.NAME, "search_query"))
)
search_box.send_keys(name)
search_button = WebDriverWait(driver, 10).until(
    EC.element_to_be_clickable((By.ID, "search-icon-legacy"))
)
search_button.click()

# 点击筛选按钮
filter_button = WebDriverWait(driver, 10).until(
    EC.element_to_be_clickable((By.CLASS_NAME, "ytd-toggle-button-renderer"))
)
filter_button.click()

# 点击筛选选项
filter_option = WebDriverWait(driver, 10).until(
    EC.element_to_be_clickable((By.CLASS_NAME, "ytd-search-filter-renderer"))
)
filter_option.click()

# 等待视频结果加载并获取链接
WebDriverWait(driver, 10).until(
    EC.presence_of_all_elements_located((By.CSS_SELECTOR, 'ytd-video-renderer h3 a'))
)
linkvideo2 = driver.find_elements(By.CSS_SELECTOR, 'ytd-video-renderer h3 a')
links = [i.get_attribute('href') for i in linkvideo2]

print(len(links))
print(driver.current_url)

driver.quit()

内容的提问来源于stack exchange，提问作者sampss s