如何用Selenium点击非按钮/锚点的More元素抓取OTC市场股票代码变更表格

Got it, let's sort out that click problem for you. The main issues with your original code are incorrect XPath syntax and not accounting for dynamic element loading. Here's how to fix it step by step:

1. Correct the Element Locator #

Your original XPath "/[contains(text()='More')]" has two critical problems:

• It doesn't specify the HTML tag (like div) you're targeting
• The contains() syntax is wrong—it should be contains(text(), "More") instead of contains(text()='More')

Since the element has a unique class _2sFaw3zGf1, you can use that to make the locator more reliable. Here are a few valid options:

• XPath: //div[@class='_2sFaw3zGf1' and contains(text(), 'More')]
• CSS Selector: div._2sFaw3zGf1 (even simpler, if the class is unique to this element)

2. Use Explicit Waits for Dynamic Content #

That "More" element is likely loaded dynamically, so you can't just click it immediately after page load. Use Selenium's WebDriverWait to wait until the element is ready to be clicked:

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By

# Wait up to 10 seconds for the element to become clickable
more_element = WebDriverWait(driver, 10).until(
    EC.element_to_be_clickable((By.XPATH, "//div[@class='_2sFaw3zGf1' and contains(text(), 'More')]"))
)
more_element.click()

3. Troubleshoot If Click Still Fails #

If the element doesn't respond to a regular click (common with non-button/anchor elements), try these workarounds:

• Scroll to the element first: Sometimes the element is off-screen, so scroll it into view before clicking:

  driver.execute_script("arguments[0].scrollIntoView({behavior: 'smooth', block: 'center'});", more_element)
  more_element.click()
  

• Trigger click via JavaScript: Bypass any UI-level restrictions by using a JavaScript click:

  driver.execute_script("arguments[0].click();", more_element)
  

4. Verify the Element is Unique #

Double-check that your locator targets only one element to avoid clicking the wrong thing:

elements = driver.find_elements(By.XPATH, "//div[@class='_2sFaw3zGf1' and contains(text(), 'More')]")
print(len(elements))  # Should output 1 if the locator is unique

Once you successfully click the "More" element, you can proceed to scrape the expanded table data with Beautiful Soup as you originally planned.

内容的提问来源于stack exchange，提问作者AmilaMGunawardana