网站下拉框选择后Python脚本无法生成JSON响应问题排查

我尝试通过Python脚本在目标网站选择三个下拉框选项并勾选复选框后获取JSON数据，但当前脚本返回415错误，具体细节如下：

原脚本 #

import requests
from bs4 import BeautifulSoup

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36',
    'Referer': 'https://info.igme.es/geoquimica/',
    'Accept': '*/*',
    'Accept-Encoding': 'gzip, deflate, br',
    'Accept-Language': 'en-US,en;q=0.9',
    'Host': 'info.igme.es',
    'Origin': 'https://info.igme.es',
}

payload = ""

with requests.Session() as s:
    s.headers.update(headers)
    s.get('https://info.igme.es/APIs/Geoquimica/Geoquimica/subsYear/2012')
    s.get('https://info.igme.es/APIs/Geoquimica/Geoquimica/subsCamp/19')
    s.get('https://info.igme.es/SpatialUnits2020/api/en/Spatial/Sheets50k/')
    resp = s.post('https://info.igme.es/APIs/Geoquimica/Geoquimica/resultYearCampSubs/500/2012/19?codSubs=1',data=payload)
    print(resp.content)

当前返回结果 #

b'{"type":"https://tools.ietf.org/html/rfc7231#section-6.5.13","title":"Unsupported Media Type","status":415,"traceId":"00-a689d50c6d8d128cb51d05781ab8c367-0fbd1e1604037fda-00"}'

期望获取的JSON输出（截断版） #

{
    "id": "51911000S1",
    "codCampana": 19,
    "codPrograma": 5,
    "nombrePrograma": "Mapa Geoquímico de España",
    "nombreCamp": "Mapa Geoquímico de España",
    "codTipoMuestra": 1,
    "tipoMuestra": "Sedimento",
    "refMuestra": "1000S1",
    "litologia": "Desconocida",
    "sustancia": "Ag",
    "concentracion": 0.06,
    "unidad": "ppm",
    "hoja": 1000,
    "huso": "30",
    "x": "164959",
    "y": "4136913",
    "scu": "Ag-0.06-ppm",
    "geometria": "37.316639,-6.781774",
    "pointWKT": "POINT (-6.781774 37.316639)",
    "xgeometria": -6.781774,
    "ygeometria": 37.316639,
    "cultivos": null,
    "sustancias": "Au | Ag | Cu | Cd | Mo | Pb | Ni | Zn | S | Al | As | Ba | Be | Bi | Br | Ca | Co | Cr | Cs | Fe | Hf | Hf | Ga | Ge | Hg | In | Ir | K | Li | Mg | Mn | Na | Nb | P | Rb | Re | Sb | Sc | Se | Sn | Sr | Ta | Te | Ti | Th | Tl | U | V | W | Y | Zr | La | La | Ce | Ce | Pr | Nd | Nd | Sm | Sm | Eu | Eu | Gd | Dy | Tb | Tb | Ho | Er | Tm | Yb | Yb | Lu | Lu | Mass | Li | Be | B | Na | Mg | Al | K | Bi | Ca | Sc | V | Cr | Mn | Fe | Co | Ni | Cu | Zn | Ga | Ge | As | Se | Rb | Sr | Y | Zr | Nb | Mo | Ag | Cd | In | Sn | Sb | Te | Cs | Ba | La | Ce | Pr | Nd | Sm | Eu | Gd | Tb | Dy | Ho | Er | Tm | Yb | Lu | Hf | Ta | W | Re | Au | Tl | Pb | Th | U | pH | TOC | ",
    "seleccion": false
}

问题原因与修复方案 #

415错误表示服务器不接受当前请求的媒体类型，原脚本存在两个关键问题：

1. 未设置Content-Type请求头，服务器无法识别请求数据格式
2. payload为空字符串，不符合API预期的JSON格式要求

修改后的脚本 #

import requests

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36',
    'Referer': 'https://info.igme.es/geoquimica/',
    'Accept': '*/*',
    'Accept-Encoding': 'gzip, deflate, br',
    'Accept-Language': 'en-US,en;q=0.9',
    'Host': 'info.igme.es',
    'Origin': 'https://info.igme.es',
    'Content-Type': 'application/json'  # 添加该头，指定数据格式为JSON
}

payload = {}  # 改为空JSON对象，替代空字符串

with requests.Session() as s:
    s.headers.update(headers)
    # 预加载必要接口
    s.get('https://info.igme.es/APIs/Geoquimica/Geoquimica/subsYear/2012')
    s.get('https://info.igme.es/APIs/Geoquimica/Geoquimica/subsCamp/19')
    s.get('https://info.igme.es/SpatialUnits2020/api/en/Spatial/Sheets50k/')
    # 使用json参数自动处理JSON格式，替代data参数
    resp = s.post('https://info.igme.es/APIs/Geoquimica/Geoquimica/resultYearCampSubs/500/2012/19?codSubs=1', json=payload)
    # 直接解析JSON并打印
    print(resp.json())

关键修改点说明 #

• 添加Content-Type: application/json头，明确告知服务器请求数据为JSON格式
• 将payload改为空JSON对象{}，符合API的数据格式要求
• 使用json=payload替代data=payload，requests会自动处理JSON序列化和头信息，避免手动格式错误

运行修改后的脚本，即可获取到期望的JSON数据。

内容的提问来源于stack exchange，提问作者MITHU