如何将API返回的字符串转换为字典/JSON对象？

问题背景 #

API调用流程正常，但返回结果开头带有特殊前缀，无法直接解析为JSON字典，导致键值对调用失败。

初始请求代码 #

with requests.Session() as s:
    resp = s.post(url, json={"user": "s153152","password": "password123"}, headers=headers, verify=False)
    print(resp.status_code)

    timestamp = None
    deleteData = False
    fields = None

    resp2 = s.post(url2, json={
        "timestamp" : timestamp,
        "deleteData" : deleteData,
        "fields":fields}, headers=headers2, verify=False, timeout=45)

    resp2 = resp2.text
    print(type(resp2))
    for data in resp['hostName']:
        print(data)

尝试解析的错误代码 #

resp3 = resp2.text
data = json.loads(resp3)
print(data)
print(type(resp2))
for i in data['hostName']:
    print(i)

报错信息 #

json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

异常返回格式 #

返回字符串开头包含JSON劫持防护前缀，具体内容如下：

")]}',
    {
"newestTimestamp": "2016-08-29T13:43:59.0000001",
"data": [
{
"hostName": "MULBERRY",
"ipAddresses": [
"192.168.146.131"
],
"locationsWithMatches": "538",
"locationsWithUnprotectedMatches": "93",
"totalMatches": "9706",
"totalUnprotectedMatches": "1319",
"lastSearchTime": "2015-08-11T18:59:45.927",
"id": -2147479361,
"changedTimestamp": "2015-08-11T18:59:45.927",
"dataTypes": [
{
"count": 914,
"name": "Social Security Number"
},
{
"count": 1237,
"name": "Credit Card Number"
}
]
},
{
"hostName": "DYCKMAN",
"ipAddresses": [
"192.168.124.184"
],
"locationsWithMatches": "592",
"locationsWithUnprotectedMatches": "93",
"totalMatches": "11470",
"totalUnprotectedMatches": "1782",
"lastSearchTime": "2015-08-31T05:53:05.28",
"id": -2147479360,
"changedTimestamp": "2015-08-31T05:53:05.28",
"dataTypes": [
{
"count": 955,
"name": "Social Security Number"
},
{
"count": 555,
"name": "Credit Card Number"
}
],
"classifications": [
{
"count": 24,
"name": "Confidential"
},
{
"count": 12,
"name": "PII"
}
]
}]
}

解决方案 #

这种前缀是API的JSON劫持防护机制，需先去除前缀再解析：

1. 定位有效JSON的起始位置（第一个{）
2. 截取有效子串后用json.loads()解析
3. 修正字段路径：hostName在data数组的元素中，而非顶级字段

修改后的代码：

import json
import requests

with requests.Session() as s:
    # 登录请求
    resp = s.post(url, json={"user": "s153152","password": "password123"}, headers=headers, verify=False)
    print(resp.status_code)

    timestamp = None
    deleteData = False
    fields = None

    # 数据请求
    resp2 = s.post(url2, json={
        "timestamp" : timestamp,
        "deleteData" : deleteData,
        "fields":fields}, headers=headers2, verify=False, timeout=45)
    
    resp_text = resp2.text
    # 去除开头的防护前缀，截取有效JSON部分
    start_index = resp_text.find('{')
    clean_json_str = resp_text[start_index:]
    # 解析为字典
    data = json.loads(clean_json_str)
    
    # 遍历并打印hostName
    for item in data['data']:
        print(item['hostName'])

内容的提问来源于stack exchange，提问作者dot_davidjb