使用Beautiful Soup提取YouTube视频信息失败的问题求助

Hey there! Let's break down why your videos variable is coming up empty and how to fix this problem.

Why This Is Happening #

The core issue here is that YouTube's page structure has changed drastically since that code was written. The class name yt-uix-tile-link hasn't been used on YouTube in years—so your BeautifulSoup selector is searching for elements that no longer exist. Even though you're successfully fetching the page content, there's nothing matching that outdated class.

Fix & Workaround Ideas #

• Inspect YouTube's current page structure
  Open YouTube's search results in your browser, right-click on a video title, and select "Inspect" to view the element's HTML. You'll notice modern YouTube uses dynamic, often random-looking class names or relies on data attributes instead. For example, video links are now nested inside elements with classes like ytd-video-renderer, and the link itself might have an attribute like id="video-title" or data-video-id. Update your BeautifulSoup selector to match this current structure.

• Handle dynamically loaded content
  YouTube loads most search results via JavaScript, which means the static HTML you get from requests.get() won't include all video elements. If you want to keep scraping, use a tool like selenium to simulate a real browser that executes JS. Here's a quick snippet to get you started:

  from selenium import webdriver
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.support import expected_conditions as EC
  from urllib.parse import quote
  
  def Play(speech):
      if speech.endswith("on YouTube"):
          search_term = " ".join(speech.split()[:-2])
          url = f"https://www.youtube.com/results?search_query={quote(search_term)}"
          
          # Initialize browser (ensure ChromeDriver is installed and in your PATH)
          driver = webdriver.Chrome()
          driver.get(url)
          
          # Wait for video elements to load
          wait = WebDriverWait(driver, 10)
          videos = wait.until(EC.presence_of_all_elements_located((By.ID, "video-title")))[:3]
          
          names = [video.text for video in videos]
          links = [video.get_attribute("href") for video in videos]
          
          print(f"I found 3 videos. {'. '.join(names)}", links)
          driver.quit()
  

• Use the official YouTube Data API (highly recommended)
  Scraping YouTube violates their Terms of Service, and frequent page changes will keep breaking your code. The most reliable long-term solution is to use the YouTube Data API v3. You'll need a free API key from Google Cloud (limited usage is free), and it gives you structured, consistent results. Here's a simplified example:

  from googleapiclient.discovery import build
  from urllib.parse import quote
  
  YOUTUBE_API_KEY = "your_api_key_here"  # Replace with your actual API key
  
  def Play(speech):
      if speech.endswith("on YouTube"):
          search_term = " ".join(speech.split()[:-2])
          
          youtube = build("youtube", "v3", developerKey=YOUTUBE_API_KEY)
          request = youtube.search().list(
              q=search_term,
              part="snippet",
              type="video",
              maxResults=3
          )
          response = request.execute()
          
          names = [item["snippet"]["title"] for item in response["items"]]
          links = [f"https://www.youtube.com/watch?v={item['id']['videoId']}" for item in response["items"]]
          
          print(f"I found 3 videos. {'. '.join(names)}", links)
  

Final Notes #

If you stick with scraping, you'll need to regularly update your selectors as YouTube tweaks its site. The API is far more stable and compliant, so it's worth setting up if you plan to maintain this voice assistant project long-term.

内容的提问来源于stack exchange，提问作者branlito