如何使用Selenium抓取网站文本？需定位未知内容“Gorgeous Geodes Glowed”

Hey there! Let's walk through how to grab that inner text containing "Gorgeous Geodes Glowed" using Python, Selenium, and even pyautogui as a fallback. Here's a step-by-step breakdown tailored to your setup:

Selenium directly interacts with the browser's DOM, so it's the best first choice for text extraction. Here's how to implement it:

• Initialize your Selenium driver
  Make sure you've got the correct browser driver (e.g., ChromeDriver for Chrome) matching your browser version.

  from selenium import webdriver
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.support import expected_conditions as EC
  
  # Launch Chrome (replace with Firefox/Edge driver if needed)
  driver = webdriver.Chrome()
  driver.get("your_target_website_url_here")
  

• Locate the element with your target text
  Use an XPath selector to find any element that contains the phrase "Gorgeous Geodes Glowed". We'll add an explicit wait to handle dynamic content loading:

  # Wait up to 10 seconds for the element to appear
  wait = WebDriverWait(driver, 10)
  target_element = wait.until(EC.presence_of_element_located(
      (By.XPATH, "//*[contains(text(), 'Gorgeous Geodes Glowed')]")
  ))
  
  # Extract the full inner text of the element
  inner_text = target_element.text
  print(inner_text)
  

  If you know the specific HTML tag (e.g., <div>, <p>), you can narrow it down for better accuracy:
  //div[contains(text(), 'Gorgeous Geodes Glowed')]

If the text is in a non-DOM element (like a canvas or a closed-shadow DOM) that Selenium can't reach, pyautogui can work as a last resort. Note: This depends on screen resolution and element position, so it's less reliable.

• Prepare the screen and focus the browser

  driver.maximize_window()
  import pyautogui
  
  # Ensure the browser window is focused
  pyautogui.click(100, 100) # Click a spot on the browser window to focus it
  

• Locate and copy the text
  Use the browser's find function to highlight the text, then select and copy it:

  # Open the browser's find bar (Ctrl+F on Windows/Linux, Cmd+F on Mac)
  pyautogui.hotkey('ctrl', 'f')
  pyautogui.typewrite('Gorgeous Geodes Glowed')
  pyautogui.press('enter')
  
  # Adjust these coordinates to match the position of your text on screen
  # You can get coordinates using pyautogui.displayMousePosition() first
  pyautogui.moveTo(x=250, y=350)
  pyautogui.dragRel(x=600, y=0, duration=1) # Drag right to select the full text line
  
  # Copy to clipboard and retrieve the text
  pyautogui.hotkey('ctrl', 'c')
  import pyperclip
  inner_text = pyperclip.paste()
  print(inner_text)
  

• Always use WebDriverWait instead of time.sleep() to handle dynamic content—it's more efficient and reliable.
• If the text is inside an iframe, switch to the iframe first with driver.switch_to.frame("iframe_id_or_name").
• For shadow DOM elements, use JavaScript execution to access the text:

  inner_text = driver.execute_script("""
      const targetElement = document.querySelector('your-element-selector');
      return targetElement.shadowRoot.textContent;
  """)
  

Hope this helps you grab that text without a hitch! If you run into issues with specific elements or dynamic loading, feel free to tweak the selectors or waits.

内容的提问来源于stack exchange，提问作者George Canon