使用Python Requests传递参数字典调用RESTful API时遇格式错误

问题情况 #

用Python的requests库调用EIA的RESTful API时，通过get方法传递嵌套结构的参数字典会返回400错误；但直接把参数拼在URL里就能正常获取数据。

报错的代码示例 #

import requests as re
url = 'https://api.eia.gov/v2/natural-gas/pri/fut/data/?api_key=' + API_KEY

params = {
    "frequency": "daily",
    "data": ["value"],
    "start": "2021-01-01",
    "facets": {"series": ["RNGWHHD"]},
    "offset": 0,
    "length": 5000
}

x = re.get(url=url, params=params)

if x.status_code == 200:
    print('Success')
else:
    print(x.text) 

返回的错误信息：

{"error":"Invalid format for facet 'series'. Must provide which facet and an array of values. For example, to filter by location for VA and CA: facets[location][]=VA&facets[location][]=CA","code":400}

可正常运行的URL拼接方式 #

url = 'https://api.eia.gov/v2/natural-gas/pri/fut/data/?frequency=daily&data[0]=value&facets[series][]=RNGWHHD&start=2021-01-01&sort[0][column]=period&sort[0][direction]=desc&offset=0&length=5000&api_key=' + API_KEY

x = re.get(url=url)

问题根源 #

requests库默认的参数编码逻辑和EIA API的格式要求不匹配：

• EIA API要求数组类型的嵌套参数必须写成facets[series][]=RNGWHHD这种带[]的格式，明确标识这是数组值
• 但requests处理{"facets": {"series": ["RNGWHHD"]}}这类嵌套字典时，会把它编码成facets[series]=RNGWHHD，缺少了末尾的[]，导致API无法识别参数格式，从而返回错误。

解决办法 #

方法1：直接构造API要求的扁平参数字典 #

把嵌套结构拆成API需要的键名格式，直接定义参数：

import requests as re
url = 'https://api.eia.gov/v2/natural-gas/pri/fut/data/?api_key=' + API_KEY

params = {
    "frequency": "daily",
    "data[]": "value",
    "start": "2021-01-01",
    "facets[series][]": "RNGWHHD",
    "offset": 0,
    "length": 5000
}

x = re.get(url=url, params=params)

if x.status_code == 200:
    print('Success')
else:
    print(x.text)

方法2：用urllib.parse手动编码参数 #

如果参数结构复杂，使用urlencode配合doseq=True生成符合要求的参数字符串：

import requests as re
from urllib.parse import urlencode

url = 'https://api.eia.gov/v2/natural-gas/pri/fut/data/?api_key=' + API_KEY

params = {
    "frequency": "daily",
    "data": ["value"],
    "start": "2021-01-01",
    "facets": {"series": ["RNGWHHD"]},
    "offset": 0,
    "length": 5000
}

# doseq=True会将列表和嵌套结构编码成API需要的带[]的格式
encoded_params = urlencode(params, doseq=True)
full_url = f"{url}&{encoded_params}"

x = re.get(full_url)

if x.status_code == 200:
    print('Success')
else:
    print(x.text)

内容的提问来源于stack exchange，提问作者Chris