如何用requests模块抓取HCAD网站指定账号的Fiduciary信息？

在HCAD网站使用账号0523620090003搜索后，可查看账号相关详情。我已用requests模块编写脚本，尝试抓取结果中的account details和fiduciary两部分内容，目前已成功抓取左上角的account details，但无法解析位于右上角中部的Fiduciary相关信息。现有脚本代码如下：

import requests
from pprint import pprint

link = 'https://arcweb.hcad.org/server/rest/services/public/public_query/MapServer/0/query'

params = {
    'f': 'json',
    'distance': 2,
    'outFields': '*',
    'outSR': '102100',
    'spatialRel': 'esriSpatialRelIntersects',
    'units': 'esriSRUnit_StatuteMile',
    'where': "HCAD_NUM = '0523620090003'",
}

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36',
    'Accept': '*/*',
    'Accept-Encoding': 'gzip, deflate, br',
    'Accept-Language': 'en-US,en;q=0.9',
}
with requests.Session() as s:
    s.headers.update(headers)
    res = s.get(link,params=params)
    pprint(res.json()['features'][0]['attributes'])

你当前调用的MapServer/0/query接口仅返回房产基础属性数据，Fiduciary（受托人）信息存储在独立的MapServer/1/query接口中。只需复用现有请求逻辑，修改接口路径即可获取该部分数据。

完整脚本如下：

import requests
from pprint import pprint

# 基础账号详情接口
account_link = 'https://arcweb.hcad.org/server/rest/services/public/public_query/MapServer/0/query'
# Fiduciary信息接口
fiduciary_link = 'https://arcweb.hcad.org/server/rest/services/public/public_query/MapServer/1/query'

params = {
    'f': 'json',
    'distance': 2,
    'outFields': '*',
    'outSR': '102100',
    'spatialRel': 'esriSpatialRelIntersects',
    'units': 'esriSRUnit_StatuteMile',
    'where': "HCAD_NUM = '0523620090003'",
}

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36',
    'Accept': '*/*',
    'Accept-Encoding': 'gzip, deflate, br',
    'Accept-Language': 'en-US,en;q=0.9',
}

with requests.Session() as s:
    s.headers.update(headers)
    # 获取账号详情
    account_res = s.get(account_link, params=params)
    print("=== Account Details ===")
    pprint(account_res.json()['features'][0]['attributes'])
    
    # 获取Fiduciary信息
    fiduciary_res = s.get(fiduciary_link, params=params)
    print("\n=== Fiduciary Information ===")
    if fiduciary_res.json()['features']:
        pprint(fiduciary_res.json()['features'][0]['attributes'])
    else:
        print("No fiduciary information found for this account.")

说明 #

• 两个接口使用完全相同的查询参数，仅路径中的MapServer/0和MapServer/1不同，分别对应不同的数据图层。
• 加入了空数据判断，避免因账号无受托人信息导致的索引错误。

内容的提问来源于stack exchange，提问作者robots.txt