如何用Python批量获取Futbin球员页面对应XHR接口的价格数据

你现有单个价格接口的player参数和球员页面路径/player/后的序号不是同一值，需要先从球员页面提取对应接口参数，再批量调用价格接口。

完整实现代码 #

import requests
import time
import csv
from lxml import etree

# 基础配置
HEADERS = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36",
    "Referer": "https://www.futbin.com/"
}
PAGE_BASE = "https://www.futbin.com/22/player/{}"
PRICE_BASE = "https://www.futbin.com/22/playerPrices?player={}"
START_ID = 1
END_ID = 10000
REQUEST_DELAY = 2  # 请求间隔，单位秒，可自行调整避免被封

# 结果存储文件初始化
with open("futbin_player_prices.csv", "w", newline="", encoding="utf-8") as f:
    writer = csv.writer(f)
    writer.writerow(["页面序号", "球员接口ID", "PC最低价格", "PS最低价格", "Xbox最低价格"])

for page_id in range(START_ID, END_ID + 1):
    try:
        # 第一步：请求球员页面提取接口需要的player_id
        page_url = PAGE_BASE.format(page_id)
        page_resp = requests.get(page_url, headers=HEADERS, timeout=10)
        if page_resp.status_code != 200:
            print(f"页面{page_id}不存在或访问失败，跳过")
            time.sleep(REQUEST_DELAY)
            continue
        
        # 提取页面中的球员资源ID，Futbin页面会把player_id放在data-player-id属性里
        html = etree.HTML(page_resp.text)
        player_id = html.xpath("//*[@data-player-id]/@data-player-id")
        if not player_id:
            print(f"页面{page_id}未提取到球员ID，跳过")
            time.sleep(REQUEST_DELAY)
            continue
        player_id = player_id[0]

        # 第二步：调用价格接口获取数据（和你提供的参考逻辑一致）
        price_resp = requests.get(PRICE_BASE.format(player_id), headers=HEADERS, timeout=10)
        price_data = price_resp.json()
        prices = price_data[player_id]["prices"]
        pc_price = prices["pc"]["LCPrice"]
        ps_price = prices["ps"]["LCPrice"]
        xbox_price = prices["xbox"]["LCPrice"]

        # 存储结果
        with open("futbin_player_prices.csv", "a", newline="", encoding="utf-8") as f:
            writer = csv.writer(f)
            writer.writerow([page_id, player_id, pc_price, ps_price, xbox_price])
        
        print(f"已获取第{page_id}号球员价格：PC={pc_price} PS={ps_price} Xbox={xbox_price}")
        time.sleep(REQUEST_DELAY)
    
    except Exception as e:
        print(f"处理第{page_id}号球员时出错：{str(e)}，跳过")
        time.sleep(REQUEST_DELAY)
        continue

使用注意事项 #

• 代码默认将结果保存在运行目录下的futbin_player_prices.csv文件中，可根据需求修改存储逻辑
• 可根据自身网络情况调整REQUEST_DELAY参数，请求间隔过短容易触发网站反爬机制导致IP被封禁
• 如果出现批量访问失败的情况，可以在HEADERS中添加你浏览器访问Futbin时的Cookie值，模拟真实用户访问
• 代码中使用了lxml库解析页面，运行前需要先执行pip install lxml安装依赖

内容的提问来源于stack exchange，提问作者HPA Informática