如何动态生成sha256Hash并用于GraphQL查询抓取房产链接？

错误原因 #

你之前的核心错误是搞反了sha256Hash的生成对象：这个哈希不是对suggest接口返回的地区ID做哈希，而是对GraphQL查询的原始文档字符串进行SHA256计算。Persisted Query的机制是客户端把查询语句的哈希发给服务器，服务器通过哈希匹配预存的查询逻辑，所以必须用正确的查询文档生成哈希。

修正步骤及脚本 #

1. 先通过浏览器抓包拿到searchByQuery对应的原始GraphQL查询文档
2. 用该文档字符串生成正确的sha256Hash
3. 将查询参数里的地区值替换为suggest接口返回的ID，确保查询目标正确

修改后的完整脚本：

import requests
import hashlib

# 原始GraphQL查询文档，必须和服务器预存的格式一致
GRAPHQL_QUERY = """
query searchByQuery($query: String!, $testListings: Boolean, $nullifyOptionals: Boolean) {
  buySearch(query: $query, testListings: $testListings, nullifyOptionals: $nullifyOptionals) {
    results {
      exact {
        items {
          listing {
            _links {
              canonical {
                href
              }
            }
          }
        }
      }
    }
  }
}
"""

def get_query_hash(query_str):
    # 清理多余空格和换行，保证和服务器端的查询文档格式完全匹配
    cleaned_query = ''.join(query_str.split())
    return hashlib.sha256(cleaned_query.encode('utf-8')).hexdigest()

url_suggest = 'https://suggest.realestate.com.au/consumer-suggest/suggestions'
url_graphql = 'https://lexa.realestate.com.au/graphql'

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36',
}
payload_suggest = {
    'max': '7',
    'type': 'suburb,region,precinct,state,postcode',
    'src': 'homepage-web',
    'query': 'Sydney CBD, NSW'
}

with requests.Session() as s:
    s.headers.update(headers)
    # 获取目标地区的官方ID
    res_suggest = s.get(url_suggest, params=payload_suggest)
    location_id = res_suggest.json()['_embedded']['suggestions'][0]['id']
    
    # 生成符合要求的查询哈希
    query_hash = get_query_hash(GRAPHQL_QUERY)
    
    # 构建GraphQL请求参数，替换地区ID和查询哈希
    params_graphql = {
        "operationName": "searchByQuery",
        "variables": {
            "query": f'{{"channel":"buy","page":1,"pageSize":25,"filters":{{"surroundingSuburbs":true,"excludeNoSalePrice":false,"ex-under-contract":false,"ex-deposit-taken":false,"excludeAuctions":false,"excludePrivateSales":false,"furnished":false,"petsAllowed":false,"hasScheduledAuction":false}},"localities":[{{"searchLocation":"{location_id}"}}]}}',
            "testListings": False,
            "nullifyOptionals": False
        },
        "extensions": {
            "persistedQuery": {
                "version": 1,
                "sha256Hash": query_hash
            }
        }
    }
    
    # 发送请求并解析结果
    res_graphql = s.post(url_graphql, json=params_graphql)
    try:
        items = res_graphql.json()['data']['buySearch']['results']['exact']['items']
        for item in items:
            print(item['listing']['_links']['canonical']['href'])
    except KeyError as e:
        print(f"请求失败，返回内容：{res_graphql.json()}")
        print(f"缺失的Key：{e}")

关键说明 #

• 必须保证GRAPHQL_QUERY的内容和服务器端预存的完全一致，包括通过清理空白来统一格式，否则哈希会不匹配
• variables里的query字符串需要用获取到的location_id替换原来的文本值，确保查询目标地区正确
• 加入了异常捕获逻辑，方便调试请求失败时的返回内容

内容的提问来源于stack exchange，提问作者robots.txt