使用Python 3.x+BeautifulSoup爬取含年龄验证弹窗网站遇阻求助

Hey there! That ElementClickInterceptedException error usually means your target age-verification button is being blocked by another element (like a modal overlay or loading screen) when you try to click it. Let's adjust your code to get around this issue.

Common Fixes to Try #

1. Use Explicit Waits Instead of time.sleep()

time.sleep(1) is unreliable because it doesn't guarantee the element is fully ready to be clicked. Instead, use WebDriverWait to wait until the button is actually interactive.

2. Use JavaScript Click if the Element is Obscured

If the button is still blocked after waiting, a JavaScript click can bypass the page's visual overlay and trigger the button directly, ignoring any on-screen obstructions.

Modified Code Implementation #

Here's your updated code with these fixes applied:

import time
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from bs4 import BeautifulSoup

# Initialize Firefox driver
driver = webdriver.Firefox(executable_path=r'/Users/jeff/Documents/geckodriver')
target_url = 'https://www.shopharborside.com/oakland/#/shop/412'
driver.get(target_url)

# Wait up to 10 seconds for the age verification button to be clickable
wait = WebDriverWait(driver, 10)
yes_button = wait.until(EC.element_to_be_clickable((By.ID, 'mBtnYes')))

# Try regular click first; if that fails, fall back to JavaScript click
try:
    yes_button.click()
except Exception as e:
    print(f"Regular click failed: {e}. Trying JavaScript click instead.")
    driver.execute_script("arguments[0].click();", yes_button)

# Wait a moment for the page to fully load after clicking
time.sleep(2)

# Grab the page source and parse with BeautifulSoup
page_source = driver.page_source
soup = BeautifulSoup(page_source, 'html.parser') # Explicit parser for consistent behavior
print(soup.prettify())

# Clean up: close the browser when done
driver.quit()

Key Notes: #

• Explicit Wait: The WebDriverWait will keep checking for the button until it's clickable (up to 10 seconds), which is way more reliable than a fixed sleep time.
• JavaScript Click: This method directly triggers the button's click event in the browser's JavaScript engine, bypassing any visual overlays that might block a regular mouse click.
• Parser Specification: Adding 'html.parser' to BeautifulSoup ensures consistent parsing across different environments.

Additional Checks: #

• Double-check that the button's ID is indeed mBtnYes (you mentioned you already located it, so this should hold, but dynamic page elements can sometimes change IDs).
• Ensure your geckodriver version matches your Firefox browser version—mismatches often cause unexpected behavior.

内容的提问来源于stack exchange，提问作者thesmeagol