使用Selenium+ChromeDriver爬取文档遇SessionNotCreatedException（页面超时）求助

Hey there, let’s work through this SessionNotCreatedException issue—since you’ve already updated Chrome and ChromeDriver to the latest versions, we can rule out the classic version mismatch problem. Here are targeted fixes to address the timeout root cause:

• Extend startup and page load timeouts
  The exception might be firing because ChromeDriver takes longer than the default timeout to spin up a session. You can adjust both the driver’s startup timeout and Chrome’s page load timeout:

  from selenium import webdriver
  from selenium.webdriver.chrome.service import Service
  
  # Set ChromeDriver startup timeout (in seconds)
  service = Service(executable_path="path/to/your/chromedriver", startup_timeout=60)
  # Add page load timeout to Chrome options
  options = webdriver.ChromeOptions()
  options.add_argument("--page-load-timeout=60")
  
  driver = webdriver.Chrome(service=service, options=options)
  

• Disable resource-heavy Chrome features to speed up startup
  Chrome’s default features (like GPU acceleration, extensions) can slow down launch and trigger timeouts. Strip these out for your crawler:

  options = webdriver.ChromeOptions()
  options.add_argument("--disable-gpu")
  options.add_argument("--disable-extensions")
  options.add_argument("--headless=new")  # Headless mode cuts down on visual overhead
  options.add_argument("--no-sandbox")  # Fixes permission issues on some systems
  options.add_argument("--disable-dev-shm-usage")  # Prevents shared memory shortages
  
  driver = webdriver.Chrome(options=options)
  

• Simulate a real browser to bypass anti-crawl delays
  Some sites intentionally delay responses to bots, causing timeouts. Add a realistic user-agent and other headers to mimic normal traffic:

  options.add_argument("user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36")
  options.add_argument("--accept-language=en-US,en;q=0.9")
  

• Use a clean Chrome profile
  Corrupted cached data can slow down Chrome’s launch. Create a temporary, fresh profile to avoid this:

  options.add_argument("--user-data-dir=/path/to/a/temporary/folder")
  options.add_argument("--disable-cache")
  

• Debug with ChromeDriver logs
  If you’re still stuck, enable driver logging to see exactly where the timeout occurs:

  service = Service(executable_path="path/to/chromedriver", log_path="chromedriver_debug.log")
  driver = webdriver.Chrome(service=service)
  

  Check the chromedriver_debug.log file for specific error messages—this will help pinpoint whether the issue is with Chrome’s launch, network delays, or site-specific blocks.

Give these steps a shot in order—most timeout-related SessionNotCreated issues get fixed by adjusting timeouts or stripping unnecessary Chrome features. If you’re still hitting walls, sharing your full error stack trace and code snippet would help narrow things down even more!

内容的提问来源于stack exchange，提问作者Neue1987