使用Selenium Wire请求航班API遭权限拒绝，如何解决？

我尝试通过代码获取网站数据，以下是我的代码，但我怀疑请求头配置存在问题，始终遭遇权限拒绝。请问该如何修改代码以解决此问题？

from seleniumwire import webdriver
from selenium.webdriver.chrome.service import Service as ChromeService
from selenium.webdriver.chrome.options import Options
from webdriver_manager.chrome import ChromeDriverManager

import time

user_agent = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36 Edg/113.0.1774.50'
sec_ch_ua = 'Microsoft Edge";v="113", "Chromium";v="113", "Not-A.Brand";v="24"'
referer = 'strict-origin-when-cross-origin'

def interceptor(request):
    # delete the "User-Agent" header and
    # set a new one
    del request.headers["user-agent"]  # Delete the header first
    request.headers["user-agent"] = user_agent
    # set the "Sec-CH-UA" header
    request.headers["sec-ch-ua"] = sec_ch_ua
    # set the "referer" header
    request.headers["referer"] = referer

Options = Options()
#Options.add_argument('--headless=new')
Path = "C:\Temp\chromedriver.exe"

driver = webdriver.Chrome(Path,
    options=Options,
    service=ChromeService(ChromeDriverManager().install())
)
# set the Selenium Wire interceptor
driver.request_interceptor = interceptor

url = "https://www.sas.dk/api/offers/flights?to=NYC&from=CPH&outDate=%2020230703&adt=1&chd=0&inf=0&yth=0&bookingFlow=points&pos=dk&channel=web&displayType=upsell"
#driver = webdriver.Chrome(Path,options=Options)
try:
    driver.get(url)

    time.sleep(5)
finally:
    driver.quit

问题分析 #

你的代码存在以下几个关键问题导致权限拒绝：

1. 转义字符错误：代码中的"是HTML转义符，Python中需替换为普通双引号"，否则会导致请求头键名或内容解析失败。
2. Referer头配置错误：strict-origin-when-cross-origin是Referrer-Policy的取值，不是Referer请求头的内容，Referer应设置为实际来源页面（如SAS官网首页）。
3. ChromeDriver初始化冲突：同时指定本地路径Path和ChromeDriverManager，二者只需保留一种，避免实例化冲突。
4. 请求头不完整：仅设置3个请求头不足以模拟真实浏览器，缺少Accept、Accept-Language等核心头。
5. 自动化特征暴露：默认Selenium配置会被网站检测为自动化工具，需禁用相关标识。
6. 方法调用错误：driver.quit缺少括号，应为driver.quit()。

修改后的代码 #

from seleniumwire import webdriver
from selenium.webdriver.chrome.service import Service as ChromeService
from selenium.webdriver.chrome.options import Options
from webdriver_manager.chrome import ChromeDriverManager
import time

# 模拟真实Edge浏览器请求头
user_agent = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36 Edg/113.0.1774.50'
sec_ch_ua = '"Microsoft Edge";v="113", "Chromium";v="113", "Not-A.Brand";v="24"'
sec_ch_ua_mobile = '?0'
sec_ch_ua_platform = '"Windows"'
referer = 'https://www.sas.dk/'
accept = 'application/json, text/plain, */*'
accept_language = 'da-DK,da;q=0.9,en-US;q=0.8,en;q=0.7'

def interceptor(request):
    # 删除原有冲突请求头
    for key in list(request.headers.keys()):
        if key.lower() in ['user-agent', 'sec-ch-ua', 'sec-ch-ua-mobile', 'sec-ch-ua-platform', 'referer', 'accept', 'accept-language']:
            del request.headers[key]
    
    # 重新设置完整请求头
    request.headers['User-Agent'] = user_agent
    request.headers['Sec-CH-UA'] = sec_ch_ua
    request.headers['Sec-CH-UA-Mobile'] = sec_ch_ua_mobile
    request.headers['Sec-CH-UA-Platform'] = sec_ch_ua_platform
    request.headers['Referer'] = referer
    request.headers['Accept'] = accept
    request.headers['Accept-Language'] = accept_language

# 配置Chrome选项，禁用自动化检测
options = Options()
# options.add_argument('--headless=new')  # 需要无头模式时启用
options.add_argument('--disable-blink-features=AutomationControlled')
options.add_argument('--disable-dev-shm-usage')
options.add_argument('--no-sandbox')
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)

# 使用ChromeDriverManager自动管理驱动，无需手动指定路径
driver = webdriver.Chrome(
    service=ChromeService(ChromeDriverManager().install()),
    options=options
)

driver.request_interceptor = interceptor

url = "https://www.sas.dk/api/offers/flights?to=NYC&from=CPH&outDate=20230703&adt=1&chd=0&inf=0&yth=0&bookingFlow=points&pos=dk&channel=web&displayType=upsell"

try:
    driver.get(url)
    time.sleep(5)
    # 可在此添加获取响应数据的逻辑，例如：
    # for req in driver.requests:
    #     if req.response:
    #         print(req.response.status_code)
    #         print(req.response.body.decode('utf-8'))
finally:
    driver.quit()

额外说明 #

• 如果仍被拒绝，可尝试先手动访问SAS官网获取有效Cookie，然后在拦截器中添加request.headers['Cookie'] = '你的Cookie内容'。
• 避免频繁请求，可适当延长等待时间或添加随机延迟，防止触发限流机制。

内容的提问来源于stack exchange，提问作者Valnurat