Selenium无法定位元素：亚特兰大房产信息地图爬取故障

尝试爬取亚特兰大房产信息地图（链接：https://gis.atlantaga.gov/propinfo/），使用Selenium定位页面中的searchInput类元素，更换114.0.5735.90和120.0.6099.62版本的ChromeDriver，以及直接用Xpath定位，均出现NoSuchElementException错误。

代码示例 #

from selenium import webdriver
import selenium
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as ec
from selenium.webdriver.common.keys import Keys
from bs4 import BeautifulSoup as bs
import time
import os


def openingDriver():
    options = webdriver.ChromeOptions()
    #options.add_argument('headless')
    options.binary_location = '{}\\{}'.format(os.getcwd(),'chromedriver.exe')
    webDriver = webdriver.Chrome(options=options)
    return webDriver

webDriver = openingDriver()
webDriver.get('https://gis.atlantaga.gov/propinfo/')

time.sleep(9)

container = webDriver.find_element(By.CLASS_NAME, 'searchInput')
container.send_keys("586 COOPER ST SW")
container.send_keys(Keys.RETURN)
time.sleep(15)

pane = webDriver.find_element(By.ID, 'cp_left')
soup = bs(pane.text, "html.parser")

错误日志 #

DevTools listening on ws://127.0.0.1:51566/devtools/browser/55df9e1b-4248-4bab-9346-b4963fe07b7b
Traceback (most recent call last):
  File "c:\Users\name\Documents\archive.py", line 24, in <module>
    container = webDriver.find_element(By.CLASS_NAME, 'searchInput')
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\name\anaconda3\envs\env\Lib\site-packages\selenium\webdriver\remote\webdriver.py", line 739, in find_element
    return self.execute(Command.FIND_ELEMENT, {"using": by, "value": value})["value"]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\name\anaconda3\envs\env\Lib\site-packages\selenium\webdriver\remote\webdriver.py", line 345, in execute
    self.error_handler.check_response(response)
  File "C:\Users\name\anaconda3\envs\env\Lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 229, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":".searchInput"}
  (Session info: chrome=120.0.6099.62); For documentation on this error, please visit: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors#no-such-element-exception
Stacktrace:
        GetHandleVerifier [0x006D6E73+174291]
        (No symbol) [0x00600AC1]
        (No symbol) [0x00316FF6]
        (No symbol) [0x00349876]
        (No symbol) [0x00349C2C]
        (No symbol) [0x0037BD42]
        (No symbol) [0x00367054]
        (No symbol) [0x0037A104]
        (No symbol) [0x00366DA6]
        (No symbol) [0x00341034]
        (No symbol) [0x00341F8D]
        GetHandleVerifier [0x007749CC+820268]
        sqlite3_dbdata_init [0x00834EBE+652494]
        sqlite3_dbdata_init [0x008348D9+650985]
        sqlite3_dbdata_init [0x0082962C+605244]
        sqlite3_dbdata_init [0x0083586B+654971]
        (No symbol) [0x0060FEBC]
        (No symbol) [0x00608428]
        (No symbol) [0x0060854D]
        (No symbol) [0x005F5858]
        BaseThreadInitThunk [0x77257BA9+25]
        RtlInitializeExceptionChain [0x77A8BD2B+107]
        RtlClearBits [0x77A8BCAF+191]

• 检查元素是否在iframe内：打开页面开发者工具，确认searchInput是否嵌套在<iframe>标签中。如果是，必须先切换到对应iframe才能定位元素：

  # 等待iframe加载并切换上下文
  WebDriverWait(webDriver, 20).until(ec.frame_to_be_available_and_switch_to_it((By.TAG_NAME, "iframe")))
  # 之后再定位目标元素
  container = WebDriverWait(webDriver, 20).until(ec.presence_of_element_located((By.CLASS_NAME, 'searchInput')))
  

• 替换固定等待为显式等待：删除time.sleep()，改用Selenium显式等待确保元素加载完成，避免网络延迟导致的定位失败：

  # 等待元素可交互后再操作
  container = WebDriverWait(webDriver, 20).until(ec.element_to_be_clickable((By.CLASS_NAME, 'searchInput')))
  

• 验证定位器准确性：

  1. 在开发者工具中用Ctrl+F搜索.searchInput，确认是否能找到唯一匹配元素；若有多个，需结合父元素属性生成更精确的定位器（如By.CSS_SELECTOR, "div.search-container .searchInput"）。
  2. 直接使用开发者工具复制的完整Xpath替换原定位方式，避免动态ID或类名的干扰。

• 修复ChromeDriver配置错误：代码中options.binary_location被错误设置为ChromeDriver路径，该参数实际用于指定Chrome浏览器的可执行文件路径。正确配置方式：

  def openingDriver():
      options = webdriver.ChromeOptions()
      # 若Chrome不在默认路径，才需设置binary_location，示例：
      # options.binary_location = "C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe"
      from selenium.webdriver.chrome.service import Service
      # 用Service类指定ChromeDriver路径
      service = Service(executable_path='chromedriver.exe')
      webDriver = webdriver.Chrome(service=service, options=options)
      return webDriver
  

• 绕过网站自动化检测：部分网站会识别Selenium特征，添加以下配置可规避检测：

  options.add_argument('--disable-blink-features=AutomationControlled')
  options.add_experimental_option("excludeSwitches", ["enable-automation"])
  options.add_experimental_option('useAutomationExtension', False)
  

内容的提问来源于stack exchange，提问作者Omar Hernandez