使用requests调用API时出现InvalidSchema错误的排查与修复

在Django项目里，我用for循环调用TMDB的TV搜索API，每次循环都会触发requests.exceptions.InvalidSchema: No connection adapters were found for错误，但奇怪的是第一个查询能正常拿到JSON结果。我怀疑是URL有多余引号或者带数字的标题导致字符串类型混乱，下面是报错栈和相关代码，求排查修复方法。

报错栈信息 #

Traceback (most recent call last):


File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/Users/tonyhall/Desktop/Coding_Stuff/pilots/script_app/views.py", line 38, in index
    response = requests.get(r)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/requests/api.py", line 76, in get
    return request('get', url, params=params, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/requests/sessions.py", line 649, in send
    adapter = self.get_adapter(url=request.url)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/requests/sessions.py", line 742, in get_adapter
    raise InvalidSchema("No connection adapters were found for {!r}".format(url))
requests.exceptions.InvalidSchema: No connection adapters were found for '{\'page\': 1, \'results\': [{\'backdrop_path\': \'/c4CSgKL6QfkJxsWcGYDyTxpbzpW.jpg\', \'first_air_date\': \'2017-03-31\', \'genre_ids\': [18, 9648], \'id\': 66788, \'name\': \'13 Reasons Why\', \'origin_country\': [\'US\'], \'original_language\': \'en\', \'original_name\': \'13 Reasons Why\', \'overview\': \"After a teenage girl\'s perplexing suicide, a classmate receives a series of tapes that unravel the mystery of her tragic choice.\", \'popularity\': 88.15, \'poster_path\': \'/nel144y4dIOdFFid6twN5mAX9Yd.jpg\', \'vote_average\': 7.7, \'vote_count\': 3489}, {\'backdrop_path\': \'/8InkNNcz5MGZUgeuZA58nQ1glGJ.jpg\', \'first_air_date\': \'2017-03-31\', \'genre_ids\': [10767], \'id\': 80865, \'name\': \'13 Reasons Why: Beyond the Reasons\', \'origin_country\': [\'US\'], \'original_language\': \'en\', \'original_name\': \'13 Reasons Why: Beyond the Reasons\', \'overview\': \'Cast members, writers, producers and mental health professionals discuss some of the difficult issues and themes explored in \"13 Reasons Why.\", \'popularity\': 10.454, \'poster_path\': \'/aauFzhjAGSemDpDBSmWDJpkeQiw.jpg\', \'vote_average\': 6.1, \'vote_count\': 21}], \'total_pages\': 1, \'total_results\': 2}'
[22/Oct/2022 04:34:36] \"GET / HTTP/1.1\" 500 101262
/Users/tonyhall/Desktop/Coding_Stuff/pilots/script_app/views.py

相关代码 #

from email.mime import base
import pprint
from pkgutil import get_data
from pydoc import pager
from unittest import result
from django.http import JsonResponse
from django.shortcuts import render, HttpResponse, redirect
from .models import *
import json
from django.core.paginator import Paginator
from django.db.models import Q
import requests
from pprint import pp, pprint
...

for lists in post:            
        data = requests.get(F'https://api.themoviedb.org/3/search/tv?api_key={api_key}&language=en-US&page=1&include_adult=false&query={lists}')

        r = data.json()
    response = requests.get(r)
    response.raise_for_status()  # raises exception when not a 2xx response
    if response.status_code != 204:
        return response.json()

排查与修复方案 #

核心错误原因 #

看报错信息里的"URL"是一个JSON字典字符串，问题出在：

1. 循环内将TMDB返回的JSON结果（字典类型）赋值给了r
2. 循环结束后错误地调用requests.get(r)，把字典当作URL发起请求，不符合URL格式直接触发InvalidSchema错误
3. 代码缩进错误，response = requests.get(r)在循环外，相当于用最后一次循环的r（字典）发起请求，必然报错

修复步骤 #

1. 修正缩进逻辑：把API响应的处理代码放到循环内部，避免循环外的无效请求
2. 删除二次请求：第一次requests.get已经拿到TMDB的返回数据，直接处理data.json()即可，不需要用结果再发起新请求
3. 处理URL编码问题：直接拼接变量到URL会出现空格、特殊字符导致的URL无效，改用requests的params参数传递查询参数，自动处理编码

修正后的代码示例 #

# 导入部分保持不变
...

def index(request):
    api_key = "你的API密钥"
    post = ["要搜索的内容1", "要搜索的内容2"]  # 假设这是你的post数据源
    results = []
    
    for item in post:            
        # 用params参数传递所有查询参数，自动处理编码
        params = {
            'api_key': api_key,
            'language': 'en-US',
            'page': 1,
            'include_adult': 'false',
            'query': item
        }
        # 发起请求
        response = requests.get('https://api.themoviedb.org/3/search/tv', params=params)
        response.raise_for_status()  # 检查请求是否成功，失败则抛出异常
        # 获取JSON结果并添加到列表中
        tv_data = response.json()
        results.append(tv_data)
    
    # 返回结果到前端
    return JsonResponse(results, safe=False)

额外注意事项 #

• 循环变量名lists易产生歧义，改成item更清晰
• 禁止直接在URL里拼接用户输入或动态内容，params参数能避免大部分编码问题
• 可添加try-except逻辑，处理网络超时、API返回错误码等异常情况

内容的提问来源于stack exchange，提问作者T_hall