使用ScrapingAnt调用Stake.com GraphQL接口遇POST请求错误求助
实时数据解析器请求错误排查与解决方案
问题描述
开发实时数据解析器时,使用ScrapingAnt绕过Cloudflare,POST请求返回错误:
POST body missing, invalid Content-Type, or JSON object has no keys.
当前使用VPN,且必须传递Cookie,代码实现如下:
headers = { "accept": "*/*", "accept-encoding": "gzip, deflate, br", "accept-language": "ru-RU,ru;q=0.9,en-GB;q=0.8,en;q=0.7,en-US;q=0.6", "cf-device-type": "", "content-length": "3315", "content-type": "application/json", "cookie": "session_info=undefined; currency_currency=btc; currency_hideZeroBalances=false; currency_currencyView=crypto; currency_bankingCurrencies=[]; casinoSearch=['Monopoly','Crazy Time','Sweet Bonanza','Money Train','Reactoonz']; sportsSearch=['Liverpool FC','Kansas City Chiefs','Los Angeles Lakers','FC Barcelona','FC Bayern Munich']; oddsFormat=decimal; sportMarketGroupMap={}; locale=ru; intercom-id-cx1ywgf2=86f79ef7-ca71-4205-8f41-b73b0b559b2e; intercom-session-cx1ywgf2=; cookie_consent=true; leftSidebarView_v2=minimized; sidebarView=hidden; cf_clearance=6420a111bb498d49b56800690b298b7bba53e91d-1667643880-0-150; __cf_bm=cj_pRlIaag.zmXOLQPWJ0GEip_W3NuRcjBa.OlOvIzU-1667643883-0-Ad3+LGxBsAD+n4k5G6mVTfRhfqAthNtU9O9VY4MicOoFQ82/DvoS6h44JXKfexV2niXlGcEBTEMB9VUOYiNbr/2tr1EidvV2unVIk7hyX8cYAcc0btV2eZv1yvPZEcGumjKYXvKuFJOx/vPpi53NXizPc8apm56HvNxb9SkKULIy", "dnt": "1", "origin": "https://stake.com", "referer": "https://stake.com/sports/home", "sec-ch-ua": "'Google Chrome';v='107', 'Chromium';v='107', 'Not=A?Brand';v='24'", "sec-ch-ua-mobile": "?0", "sec-ch-ua-platform": "Windows", "sec-fetch-dest": "empty", "sec-fetch-mode": "cors", "sec-fetch-site": "same-origin", "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36", "x-forwarded-for": "88.99.58.45, 162.158.38.53, 172.20.242.28", "x-geoip-country": "DE", "x-geoip-state": "", "x-language": "ru" } body = """ query highrollerSportBets($limit: Int!) { highrollerSportBets(limit: $limit) { ...RealtimeSportBet __typename } } fragment RealtimeSportBet on Bet { id iid bet { __typename ... on PlayerPropBet { ...PlayerPropBetFragment __typename } ... on SportBet { outcomes { fixture { data { __typename ... on SportFixtureDataMatch { competitors { name abbreviation __typename } __typename } } tournament { category { sport { slug __typename } __typename } __typename } __typename } __typename } createdAt potentialMultiplier amount currency user { id name __typename } __typename } } } fragment PlayerPropBetFragment on PlayerPropBet { __typename active amount cashoutMultiplier createdAt currency customBet id odds payout payoutMultiplier updatedAt status user { id name __typename } playerProps { id odds lineType playerProp { ...PlayerPropLineFragment __typename } __typename } } fragment PlayerPropLineFragment on PlayerPropLine { id line over under suspended balanced name player { id name __typename } market { id stat { name value __typename } game { id fixture { id name status eventStatus { ...FixtureEventStatus __typename } data { ... on SportFixtureDataMatch { __typename startTime competitors { ...CompetitorFragment __typename } } __typename } tournament { id category { id sport { id name slug __typename } __typename } __typename } __typename } __typename } __typename } } fragment FixtureEventStatus on SportFixtureEventStatus { homeScore awayScore matchStatus clock { matchTime remainingTime __typename } periodScores { homeScore awayScore matchStatus __typename } currentServer { extId __typename } homeGameScore awayGameScore statistic { yellowCards { away home __typename } redCards { away home __typename } corners { home away __typename } __typename } } fragment CompetitorFragment on SportFixtureCompetitor { name extId countryCode abbreviation } """ operationName = "highrollerSportBets" variables = {"limit":10} url = 'https://stake.com/_api/graphql' sa_key = '280a2b7336344a8ea15106dd3220cc5a' sa_api = 'https://api.scrapingant.com/v2/general' qParams = {'url': url, 'x-api-key': sa_key} reqUrl = f'{sa_api}?{urllib.parse.urlencode(qParams)}' r = requests.post(url=reqUrl, json={"query": body, "operationName": operationName, "variables": variables}, headers=headers) print(r.text)
错误原因分析
- ScrapingAnt API请求格式错误:直接将GraphQL请求体传给ScrapingAnt,但ScrapingAnt的POST接口要求将目标请求的参数(方法、数据、请求头等)封装到特定结构中,而非直接传递目标接口的请求体。
- 硬编码
content-length:手动设置的content-length值与实际请求体长度不匹配,导致服务器解析失败。 - VPN与ScrapingAnt冲突:ScrapingAnt自带代理IP能力,同时使用VPN会造成IP混淆,干扰Cloudflare验证和请求路由。
- Cookie传递方式问题:直接在请求头中传递Cookie,可能因ScrapingAnt的转发机制导致Cookie未正确送达目标服务器。
修正方案
1. 调整ScrapingAnt请求格式
修正后的代码将目标请求的所有参数封装到ScrapingAnt要求的请求体结构中:
import requests import urllib.parse headers = { "accept": "*/*", "accept-encoding": "gzip, deflate, br", "accept-language": "ru-RU,ru;q=0.9,en-GB;q=0.8,en;q=0.7,en-US;q=0.6", "cf-device-type": "", "content-type": "application/json", "cookie": "session_info=undefined; currency_currency=btc; currency_hideZeroBalances=false; currency_currencyView=crypto; currency_bankingCurrencies=[]; casinoSearch=['Monopoly','Crazy Time','Sweet Bonanza','Money Train','Reactoonz']; sportsSearch=['Liverpool FC','Kansas City Chiefs','Los Angeles Lakers','FC Barcelona','FC Bayern Munich']; oddsFormat=decimal; sportMarketGroupMap={}; locale=ru; intercom-id-cx1ywgf2=86f79ef7-ca71-4205-8f41-b73b0b559b2e; intercom-session-cx1ywgf2=; cookie_consent=true; leftSidebarView_v2=minimized; sidebarView=hidden; cf_clearance=6420a111bb498d49b56800690b298b7bba53e91d-1667643880-0-150; __cf_bm=cj_pRlIaag.zmXOLQPWJ0GEip_W3NuRcjBa.OlOvIzU-1667643883-0-Ad3+LGxBsAD+n4k5G6mVTfRhfqAthNtU9O9VY4MicOoFQ82/DvoS6h44JXKfexV2niXlGcEBTEMB9VUOYiNbr/2tr1EidvV2unVIk7hyX8cYAcc0btV2eZv1yvPZEcGumjKYXvKuFJOx/vPpi53NXizPc8apm56HvNxb9SkKULIy", "dnt": "1", "origin": "https://stake.com", "referer": "https://stake.com/sports/home", "sec-ch-ua": "'Google Chrome';v='107', 'Chromium';v='107', 'Not=A?Brand';v='24'", "sec-ch-ua-mobile": "?0", "sec-ch-ua-platform": "Windows", "sec-fetch-dest": "empty", "sec-fetch-mode": "cors", "sec-fetch-site": "same-origin", "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36", "x-forwarded-for": "88.99.58.45, 162.158.38.53, 172.20.242.28", "x-geoip-country": "DE", "x-geoip-state": "", "x-language": "ru" } # 移除硬编码的content-length,由requests自动计算 del headers["content-length"] body = """ query highrollerSportBets($limit: Int!) { highrollerSportBets(limit: $limit) { ...RealtimeSportBet __typename } } fragment RealtimeSportBet on Bet { id iid bet { __typename ... on PlayerPropBet { ...PlayerPropBetFragment __typename } ... on SportBet { outcomes { fixture { data { __typename ... on SportFixtureDataMatch { competitors { name abbreviation __typename } __typename } } tournament { category { sport { slug __typename } __typename } __typename } __typename } __typename } createdAt potentialMultiplier amount currency user { id name __typename } __typename } } } fragment PlayerPropBetFragment on PlayerPropBet { __typename active amount cashoutMultiplier createdAt currency customBet id odds payout payoutMultiplier updatedAt status user { id name __typename } playerProps { id odds lineType playerProp { ...PlayerPropLineFragment __typename } __typename } } fragment PlayerPropLineFragment on PlayerPropLine { id line over under suspended balanced name player { id name __typename } market { id stat { name value __typename } game { id fixture { id name status eventStatus { ...FixtureEventStatus __typename } data { ... on SportFixtureDataMatch { __typename startTime competitors { ...CompetitorFragment __typename } } __typename } tournament { id category { id sport { id name slug __typename } __typename } __typename } __typename } __typename } __typename } } fragment FixtureEventStatus on SportFixtureEventStatus { homeScore awayScore matchStatus clock { matchTime remainingTime __typename } periodScores { homeScore awayScore matchStatus __typename } currentServer { extId __typename } homeGameScore awayGameScore statistic { yellowCards { away home __typename } redCards { away home __typename } corners { home away __typename } __typename } } fragment CompetitorFragment on SportFixtureCompetitor { name extId countryCode abbreviation } """ operationName = "highrollerSportBets" variables = {"limit":10} target_url = 'https://stake.com/_api/graphql' sa_key = '280a2b7336344a8ea15106dd3220cc5a' sa_api = 'https://api.scrapingant.com/v2/general' # 构造ScrapingAnt标准请求体 sa_payload = { "url": target_url, "method": "POST", "headers": headers, "data": { "query": body.strip(), "operationName": operationName, "variables": variables }, "x-api-key": sa_key } # 关闭VPN,使用ScrapingAnt自带代理 r = requests.post(url=sa_api, json=sa_payload) print(r.text)
2. 其他关键调整
- 移除硬编码的
content-length,让requests库自动计算并设置该值。 - 对GraphQL查询语句执行
strip()处理,去除多余空白字符避免格式问题。 - 建议关闭VPN,使用ScrapingAnt自带代理IP,避免IP冲突干扰Cloudflare验证。
适用工具与库
- ScrapingAnt官方SDK:简化请求构造逻辑,减少格式错误概率。
- Playwright/Puppeteer:模拟真实浏览器行为,应对复杂JavaScript渲染和Cloudflare人机验证场景。
- Cloudscraper:专门针对Cloudflare的Python库,自动处理挑战页面,适合简单请求场景。
- Scrapy + ScrapingAnt中间件:适合大规模爬取需求,高效处理请求转发与Cloudflare绕过。
内容的提问来源于stack exchange,提问作者Perrk
相关产品推荐
相关产品推荐

