Google Maps爬虫clickAcceptAllButton函数TimeoutException故障求助

问题情况 #

我在用Selenium爬取Google Maps数据时，clickAcceptAllButton函数触发了selenium.common.exceptions.TimeoutException，尝试延长等待时间、最大化窗口都无法解决。

原核心代码 #

def clickAcceptAllButton():
    """
    This function is responsible for clicking "accept all" button at the first page opened after initialization of
    the webdriver.
    """
    global googleAcceptButtonClicked
    button_path = '//*[@id="yDmH0d"]/c-wiz/div/div/div/div[2]/div[1]/div[3]/div[1]/div[1]/form[2]/div/div/button'
    wait = WebDriverWait(driver, 10)
    button = wait.until(expected_conditions.presence_of_element_located((By.XPATH, button_path)))
    button.click()
    googleAcceptButtonClicked = True

报错信息 #

(base) jdtorres@SIE-C02G50GLML86 Google-Maps-Scraper-master % /Users/jdtorres/opt/anaconda3/bin/python /Users/jdtorres/Downloads/Google-Maps-Scraper-master/MainScraper.py
/Users/jdtorres/Downloads/Google-Maps-Scraper-master/MainScraper.py:21: DeprecationWarning: executable_path has been deprecated, please pass in a Service object
  driver = webdriver.Chrome(ChromeDriverManager().install())
['bar', 'cinema', 'office']
total number of points to check:175
Traceback (most recent call last):
  File "/Users/jdtorres/Downloads/Google-Maps-Scraper-master/MainScraper.py", line 183, in <module>
    new_places = searchForPlace(url, typeOfPlace)
  File "/Users/jdtorres/Downloads/Google-Maps-Scraper-master/MainScraper.py", line 72, in searchForPlace
    clickAcceptAllButton()
  File "/Users/jdtorres/Downloads/Google-Maps-Scraper-master/MainScraper.py", line 101, in clickAcceptAllButton
    button = wait.until(EC.visibility_of_element_located((By.XPATH, button_path)))
  File "/Users/jdtorres/opt/anaconda3/lib/python3.8/site-packages/selenium/webdriver/support/wait.py", line 95, in until
    raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:
Stacktrace:
0   chromedriver                        0x000000010d740598 chromedriver + 4973976
1   chromedriver                        0x000000010d737913 chromedriver + 4938003
2   chromedriver                        0x000000010d2f4cd6 chromedriver + 470230
3   chromedriver                        0x000000010d3382ff chromedriver + 746239
4   chromedriver                        0x000000010d338591 chromedriver + 746897
5   chromedriver                        0x000000010d37a384 chromedriver + 1016708
6   chromedriver                        0x000000010d35c9ed chromedriver + 895469
7   chromedriver                        0x000000010d377856 chromedriver + 1005654
8   chromedriver                        0x000000010d35c793 chromedriver + 894867
9   chromedriver                        0x000000010d32a21f chromedriver + 688671
10  chromedriver                        0x000000010d32b56e chromedriver + 693614
11  chromedriver                        0x000000010d6fabb7 chromedriver + 4688823
12  chromedriver                        0x000000010d6ffa51 chromedriver + 4708945
13  chromedriver                        0x000000010d70640f chromedriver + 4736015
14  chromedriver                        0x000000010d70091a chromedriver + 4712730
15  chromedriver                        0x000000010d6d387c chromedriver + 4528252
16  chromedriver                        0x000000010d71ff28 chromedriver + 4841256
17  chromedriver                        0x000000010d7200a7 chromedriver + 4841639
18  chromedriver                        0x000000010d73128f chromedriver + 4911759
19  libsystem_pthread.dylib             0x00007ff816b111d3 _pthread_start + 125
20  libsystem_pthread.dylib             0x00007ff816b0cbd3 thread_start + 15

问题根源 #

1. 硬编码绝对XPATH失效：Google Maps页面结构频繁更新，绝对XPATH会因元素层级变化找不到按钮。
2. 等待条件不合理：原代码仅等待元素存在/可见，但元素可能未加载完成或被遮挡，导致无法点击。
3. 未处理按钮不存在场景：若浏览器已保存Cookie同意状态，弹窗不会出现，等待按钮会直接超时。

修复方案 #

1. 替换为稳定的定位方式 #

放弃绝对XPATH，改用基于按钮文本或通用属性的相对定位，支持多语言场景（如英文"Accept all"、波兰语"Zaakceptuj wszystko"）。

2. 添加异常处理 #

避免因按钮不存在导致脚本崩溃，直接标记为已同意并继续执行。

3. 优化等待逻辑 #

等待元素可点击而非仅存在/可见，确保操作有效。

4. 可选：初始化时禁用Cookie弹窗 #

通过Chrome配置直接跳过Cookie同意步骤，从根源避免处理弹窗。

修改后的代码 #

优化后的clickAcceptAllButton函数 #

from selenium.common.exceptions import TimeoutException

def clickAcceptAllButton():
    """
    处理Google Maps的Cookie同意弹窗
    """
    global googleAcceptButtonClicked
    try:
        wait = WebDriverWait(driver, 10)
        # 匹配多语言的"同意所有"按钮，优先等待元素可点击
        button = wait.until(EC.element_to_be_clickable(
            (By.XPATH, '//button[contains(text(), "Accept all") or contains(text(), "Zaakceptuj wszystko")]')
        ))
        button.click()
        googleAcceptButtonClicked = True
        print("已点击Cookie同意按钮")
    except TimeoutException:
        # 未找到按钮，可能已同意过或页面结构变化，直接标记为已处理
        print("未找到Cookie同意按钮，跳过")
        googleAcceptButtonClicked = True

带Cookie禁用配置的Driver初始化 #

options = webdriver.ChromeOptions()
# 禁用通知和弹窗拦截
options.add_argument("--disable-notifications")
options.add_argument("--disable-popup-blocking")
# 配置自动同意Cookie，避免弹窗出现
options.add_experimental_option("prefs", {
    "profile.default_content_setting_values.cookies": 1,
    "profile.cookie_controls_mode": 0  # 0表示允许所有Cookie
})

service = Service(ChromeDriverManager(chrome_type=ChromeType.CHROMIUM).install())
os.chmod(service.path, 755)
driver = webdriver.Chrome(service=service, options=options)
action = ActionChains(driver)

额外注意事项 #

• 定期检查定位器：Google Maps页面结构会频繁更新，若再次出现问题，需重新获取按钮的定位信息。
• 避免绝对路径：所有元素定位尽量使用相对XPATH、CSS选择器或文本匹配，减少页面变化带来的影响。
• 多场景兼容：如果爬虫需要支持不同地区，需补充对应语言的按钮文本到定位表达式中。

内容的提问来源于stack exchange，提问作者Juan David Torres