将cURL转为Python代码后请求返回403 Forbidden的原因排查

我尝试用Python的requests库爬取Betway比利时甲级联赛页面（https://sports.betway.be/nl/sports/grp/soccer/belgium/first-division-a），通过浏览器开发者工具的网络面板找到了获取赛事JSON数据的GetEvents接口，并用curlconverter将该接口的cURL请求转换为如下Python代码：

import requests

cookies = {
    '_gcl_au': '1.1.1661658686.1660753544',
    '_ga': 'GA1.2.1596995754.1660753545',
    'BETWAY_ENSIGHTEN_PRIVACY_Marketing': '1',
    'BETWAY_ENSIGHTEN_PRIVACY_Analytics': '1',
    'bwui_cookieToastDismissed': 'true',
    'ssc_DeviceId': '0bccd9bb-9f38-4ceb-b225-2956e2163d27',
    'ssc_DeviceId_HttpOnly': '0bccd9bb-9f38-4ceb-b225-2956e2163d27',
    'ai_user': 'v0JA/|2022-08-17T16:25:54.617Z',
    'ens_firstVisit': '1660753692684',
    'bw_BrowserId': '19353358536559595037829240027349442244',
    '_sp_srt_id.c606': '543c5eed-4191-4fe4-9a12-2290ac66f159.1660753575.6.1661113080.1661108591.d6222a58-ed29-4651-ac7e-0b69163cf243',
    'userLanguage': 'nl',
    'SpinSportVisitId': '33f030d9-79ab-45de-b0bd-1ef52ae71f37',
    'ssc_btag': '13d5a754-1cb1-45ec-88a5-a4ab3da0a288',
    'TrackingVisitId': '13d5a754-1cb1-45ec-88a5-a4ab3da0a288',
    'bw_SessionId': '7fab38de-b50f-4d35-8333-e86a4a026a6f',
    'ai_session': 'hkBwK|1661929976364.3|1661929976364.3',
    'domainCookie': 'betway.be',
    '_gid': 'GA1.2.2070786832.1661929989',
    '_gat_UA-1515961-1': '1',
    'TimezoneOffset': '120',
    '_gat': '1',
    '_scid': '6e81eb56-b82b-4086-b7a0-783de175b7d8',
    'ens_firstPageView': 'false',
    'AMCVS_74756B615BE2FD4A0A495EB8%40AdobeOrg': '1',
    'AMCV_74756B615BE2FD4A0A495EB8%40AdobeOrg': '359503849%7CMCIDTS%7C19236%7CMCMID%7C25582052571887037000237579884308421467%7CMCAAMLH-1662534800%7C6%7CMCAAMB-1662534800%7CRKhpRz8krg2tLO6pguXWp5olkAcUniQYPHaMWWgdJ3xzPWQmdj0y%7CMCCIDH%7C1868381894%7CMCOPTOUT-1661937200s%7CNONE%7CMCAID%7CNONE%7CvVersion%7C5.0.1',
    '_gat_reg1': '1',
    '_gat_ens': '1',
    'gpv_pn': 'nl%3Asports%3Agrp%3Asoccer%3Abelgium%3Afirst-division-a',
    's_cc': 'true',
    'StaticResourcesVersion': '12.67.0.1',
    '__cf_bm': 'ljcKg_UW1mTQeV80dsFzYz8bMMSdjFYNyXvAegwnB9c-1661930020-0-AZzywRq0XM6JfSMfWMZwpLg7AkUfHOh2ZOqCgf6xeJOmLuat6hyWkt/33xjyKT5yMZAPtLUM/OpG39S+/RtEOUg=',
}

headers = {
    'authority': 'sports.betway.be',
    'accept': 'application/json; charset=UTF-8',
    'accept-language': 'nl-NL,nl;q=0.9,en-US;q=0.8,en;q=0.7',
    'content-type': 'application/json; charset=UTF-8',
    # Requests sorts cookies= alphabetically #
    'cookie': '_gcl_au=1.1.1661658686.1660753544; _ga=GA1.2.1596995754.1660753545) and it's giving me this codeeting=1; BETWAY_ENSIGHTEN_PRIVACY_Analytics=1; bwui_cookieToastDismissed=true; ssc_DeviceId=0bccd9bb-9f38-4ceb-b225-2956e2163d27; ssc_DeviceId_HttpOnly=0bccd9bb-9f38-4ceb-b225-2956e2163d27; ai_user=v0JA/|2022-08-17T16:25:54.617Z; ens_firstVisit=1660753692684; bw_BrowserId=19353358536559595037829240027349442244; _sp_srt_id.c606=543c5eed-4191-4fe4-9a12-2290ac66f159.1660753575.6.1661113080.1661108591.d6222a58-ed29-4651-ac7e-0b69163cf243; userLanguage=nl; SpinSportVisitId=33f030d9-79ab-45de-b0bd-1ef52ae71f37; ssc_btag=13d5a754-1cb1-45ec-88a5-a4ab3da0a288; TrackingVisitId=13d5a754-1cb1-45ec-88a5-a4ab3da0a288; bw_SessionId=7fab38de-b50f-4d35-8333-e86a4a026a6f; ai_session=hkBwK|1661929976364.3|1661929976364.3; domainCookie=betway.be; _gid=GA1.2.2070786832.1661929989; _gat_UA-1515961-1=1; TimezoneOffset=120; _gat=1; _scid=6e81eb56-b82b-4086-b7a0-783de175b7d8; ens_firstPageView=false; AMCVS_74756B615BE2FD4A0A495EB8%40AdobeOrg=1; AMCV_74756B615BE2FD4A0A495EB8%40AdobeOrg=359503849%7CMCIDTS%7C19236%7CMCMID%7C25582052571887037000237579884308421467%7CMCAAMLH-1662534800%7C6%7CMCAAMB-1662534800%7CRKhpRz8krg2tLO6pguXWp5olkAcUniQYPHaMWWgdJ3xzPWQmdj0y%7CMCCIDH%7C1868381894%7CMCOPTOUT-1661937200s%7CNONE%7CMCAID%7CNONE%7CvVersion%7C5.0.1; _gat_reg1=1; _gat_ens=1; gpv_pn=nl%3Asports%3Agrp%3Asoccer%3Abelgium%3Afirst-division-a; s_cc=true; StaticResourcesVersion=12.67.0.1; __cf_bm=ljcKg_UW1mTQeV80dsFzYz8bMMSdjFYNyXvAegwnB9c-1661930020-0-AZzywRq0XM6JfSMfWMZwpLg7AkUfHOh2ZOqCgf6xeJOmLuat6hyWkt/33xjyKT5yMZAPtLUM/OpG39S+/RtEOUg=',
    'origin': 'https://sports.betway.be',
    'referer': 'https://sports.betway.be/nl/sports/grp/soccer/belgium/first-division-a',
    'sec-ch-ua': '"Chromium";v="104", " Not A;Brand";v="99", "Google Chrome";v="104"',
    'sec-ch-ua-mobile': '?0',
    'sec-ch-ua-platform': '"Windows"',
    'sec-fetch-dest': 'empty',
    'sec-fetch-mode': 'cors',
    'sec-fetch-site': 'same-origin',
    'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36',
}

json_data = {
    'LanguageId': 8,
    'ClientTypeId': 2,
    'BrandId': 3,
    'JurisdictionId': 3,
    'ClientIntegratorId': 1,
    'ExternalIds': [
        10121019,
        10121020,
        10121021,
    ],
    'MarketCName': 'win-draw-win',
    'ScoreboardRequest': {
        'ScoreboardType': 3,
        'IncidentRequest': {},
    },
    'BrowserId': 3,
    'OsId': 3,
    'ApplicationVersion': '',
    'BrowserVersion': '104.0.0.0',
    'OsVersion': 'NT 10.0',
    'SessionId': None,
    'TerritoryId': 21,
    'CorrelationId': '625f145c-8a5e-40a5-af44-a0a13116961c',
    'VisitId': '33f030d9-79ab-45de-b0bd-1ef52ae71f37',
    'ViewName': 'sports',
    'JourneyId': 'd857f2d0-b79f-4346-9683-f61d1e0c9854',
}

response = requests.post('https://sports.betway.be/api/Events/v2/GetEvents', cookies=cookies, headers=headers, json=json_data)

但运行这段代码后，请求返回403 Forbidden状态码，请问这是什么原因？

常见原因及解决方向 #

• Cookie过期或无效：代码中的Cookie是浏览器会话的临时凭证，这类Cookie通常和特定会话、设备绑定，一旦浏览器关闭或过了有效期就会失效。解决方法是重新从浏览器获取最新Cookie，或者先请求目标页面获取有效Cookie后再调用接口。
• 请求头Cookie重复传递：代码在headers里保留了cookie字段，同时又通过cookies参数传递Cookie，requests会自动处理Cookie，重复传递会导致格式冲突，服务器验证失败。建议删除headers中的cookie字段，仅保留cookies参数。
• 会话标识字段失效：请求体json_data中的CorrelationId、VisitId、JourneyId都是和当前浏览器会话绑定的唯一标识，这些值过期后服务器会判定请求非法。需要从浏览器最新的请求中提取这些字段的有效值，或者生成符合格式的UUID替换。
• Cloudflare反爬拦截：请求中的__cf_bm是Cloudflare的临时验证Cookie，Cloudflare会检测请求的行为特征（如UA一致性、请求频率、是否有浏览器环境），非浏览器发起的请求易被拦截。可以尝试用selenium或playwright模拟真实浏览器，或者确保请求头的sec-ch-ua等字段格式正确。
• 请求参数不匹配：服务器可能验证请求体所有参数的关联性，比如SessionId设为None是否符合要求，或者参数值需和Cookie中的对应值匹配。需要对比浏览器中的原始请求体，确保所有参数取值和格式一致。

内容的提问来源于stack exchange，提问作者Stef Renneboog