Selenium无头浏览器模式下页面源码显示Unavailable问题求助

我正在使用Python实现自动化流程，通过用户名和密码访问网站并检查预约选项是否可用。使用Selenium 4.11.2时，注释掉headless选项代码运行正常，但启用headless模式后，登录成功的页面源码仅显示文本Unavailable，无法获取正常的页面内容。

原始代码 #

from dotenv import load_dotenv
import os
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
import time

load_dotenv()

# Replace these with actual values
login_url = 'https://prenotami.esteri.it/Services/Booking/2391'
login = os.getenv('LOGIN')
password = os.getenv('PASSWORD')
search_text = "Stante l'elevata richiesta i posti disponibili per il servizio scelto sono esauriti."

# Set up Chrome options for headless mode
chrome_options = Options()
# chrome_options.add_argument("--headless")

# Initialize the Selenium webdriver with the configured options
driver = webdriver.Chrome(options=chrome_options)
driver.implicitly_wait(3)

# Open the login page
driver.get(login_url)
driver.get_screenshot_as_file("1_login_page.png")

# Find the login and password input fields and fill them
login_field = driver.find_element('name', 'Email')
password_field = driver.find_element('name', 'Password')
login_field.send_keys(login)
password_field.send_keys(password)

# Submit the form (you can replace this with actual form submission method if needed)
password_field.send_keys(Keys.RETURN)

# Allow some time for the page to load and the login to complete
time.sleep(3)
driver.get_screenshot_as_file("2_message_page.png")

# Check page message
if search_text in driver.page_source:
    print('Closed appointment')
else:
    print("Open appointment")

# Close the browser when done
driver.quit()

我已尝试添加Windows/Linux版本的User-Agent、多种Chrome配置、优化等待机制、设置it语言等方案，但均无效。

请问我遗漏了什么？是否无头进程被系统拦截？若是，还有其他可测试的解决方向吗？

解决方向建议 #

• 使用新版无头模式：Chrome 112+推出的--headless=new模式更接近真实浏览器行为，替换旧版--headless参数，同时添加更多模拟真实环境的配置：

  chrome_options = Options()
  chrome_options.add_argument("--headless=new")
  chrome_options.add_argument("--window-size=1920,1080")
  chrome_options.add_argument("--disable-blink-features=AutomationControlled")
  chrome_options.add_experimental_option("excludeSwitches", ["enable-automation"])
  chrome_options.add_experimental_option('useAutomationExtension', False)
  

• 隐藏自动化特征：网站可能通过navigator.webdriver属性识别自动化工具，执行JS代码隐藏该特征：

  driver.execute_script("Object.defineProperty(navigator, 'webdriver', {get: () => undefined})")
  

• 同步请求头与真实浏览器：对比正常浏览器和无头模式的请求头，手动添加关键字段到Chrome配置：

  chrome_options.add_argument("accept-language=it-IT,it;q=0.9")
  chrome_options.add_argument("referer=https://prenotami.esteri.it/")
  

• 优化页面等待逻辑：替换固定time.sleep()为显式等待，确保页面元素完全加载：

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.support import expected_conditions as EC
  from selenium.webdriver.common.by import By
  
  # 等待登录后页面加载完成
  WebDriverWait(driver, 15).until(EC.presence_of_element_located((By.TAG_NAME, "body")))
  

• 尝试替代工具：改用Playwright，它默认的无头模式隐蔽性更强，对反爬机制兼容性更好：

  from playwright.sync_api import sync_playwright
  
  with sync_playwright() as p:
      browser = p.chromium.launch(headless=True)
      page = browser.new_page()
      page.goto(login_url)
      # 后续登录逻辑...
      browser.close()
  

• 验证会话Cookie：在正常浏览器登录后导出Cookie，无头模式启动后导入Cookie，绕过登录流程直接访问目标页面，排查是否是登录环节被拦截：

  # 示例：导入预先导出的Cookie
  cookies = [{"name": "cookie_name", "value": "cookie_value", "domain": "prenotami.esteri.it"}]
  for cookie in cookies:
      driver.add_cookie(cookie)
  driver.refresh()
  

内容的提问来源于stack exchange，提问作者Gabriel Braico Dornas