Azure函数App与Django请求JSON编码不一致问题解决咨询

问题背景 #

我有一个Django应用，其中包含调用API并将响应数据存入数据库的函数。迁移到Azure函数App后，请求返回的JSON编码出现差异，导致数据库插入操作失败。

代码对比 #

Django中的请求函数 #

from requests import get, HTTPError, RequestException
import time

def fetch_magic_set_data(code, max_retries=3, sleep_base=2):
  for retry in range(max_retries):
    try:
      response = get(f'https://mtgjson.com/api/v5/{code}.json')
      response.raise_for_status()
      magic_sets_json = response.json()
      if 'data' in magic_sets_json and 'cards' in magic_sets_json['data']:
        return True, magic_sets_json['data']
      else:
        print(f'Response is empty for {code}, retrying...')
    except HTTPError as http_err:
      print(f'HTTP error occurred: {http_err}')
    except RequestException as req_err:
      print(f'Request error occurred: {req_err}')
    sleep_time = sleep_base * (2 ** retry)
    time.sleep(sleep_time)
  return False, None

Azure函数App中的请求函数 #

import requests
import time
import logging

logger = logging.getLogger(__name__)
MTGJSON_API_BASE_URL = 'https://mtgjson.com/api/v5/'

def get_response(url, max_attempts=10):
    for attempt in range(max_attempts):
        try:
            response = requests.get(url)
            if response.status_code == 200:
                return response
            elif 400 <= response.status_code < 600:
                logger.warning(f"Status: {response.status_code}. Error from {url}.")
                if response.status_code == 404:
                    return None
        except requests.RequestException as e:
            logger.warning(f"Attempt {attempt + 1} of {max_attempts}. Error: {e}")
            if attempt < max_attempts - 1:
                time.sleep((2 ** attempt) + (attempt % 2))
            else:
                logger.error(f"Failed to get response from {url} after {max_attempts} attempts.")
                return None
    return None

def fetch_magic_set_data(code):
    url = MTGJSON_API_BASE_URL + code.upper() + '.json'
    response = get_response(url)
    if response:
        try:
            magic_set_data = response.json()
            return magic_set_data['data']
        except ValueError:
            logger.error(f"Failed to parse JSON data from {url}.")
    else:
        logger.error(f"Failed to get a valid response from {url}.")
    return {}

现象对比 #

• 两者调用print(response.headers['Content-Type'])均返回application/json。

• 调用print(response.content)时：

  • Django应用返回片段：'type': 'Legendary Planeswalker — Jace',
  • Azure函数App返回片段："type": "Legendary Planeswalker \xe2\x80\x94 Jace",

• 在Azure函数App中手动解码后：

  decoded_content = response.content.decode('utf-8')
  print(decoded_content)
  magic_sets_json = json.loads(decoded_content)
  

  输出显示为："type": "Legendary Planeswalker ù Jace",

• 环境一致：两个应用均使用python version 3.10.11和requests version 2.31.0。

解决方案 #

1. 强制指定响应编码为UTF-8 #

在Azure函数的get_response函数中，获取响应后直接设置编码为UTF-8，确保解码正确：

def get_response(url, max_attempts=10):
    for attempt in range(max_attempts):
        try:
            response = requests.get(url)
            # 强制设置响应编码为UTF-8
            response.encoding = 'utf-8'
            if response.status_code == 200:
                return response
            # 其余代码保持不变
        except requests.RequestException as e:
            # 异常处理代码保持不变
    return None

设置response.encoding = 'utf-8'后，response.json()会使用UTF-8解码内容，避免编码识别错误。

2. 手动指定解码方式解析JSON #

如果设置编码仍无效，可手动读取响应内容并指定UTF-8解码后再解析：

import json

def fetch_magic_set_data(code):
    url = MTGJSON_API_BASE_URL + code.upper() + '.json'
    response = get_response(url)
    if response:
        try:
            # 手动指定UTF-8解码后解析JSON
            magic_set_data = json.loads(response.content.decode('utf-8'))
            return magic_set_data['data']
        except ValueError:
            logger.error(f"Failed to parse JSON data from {url}.")
    else:
        logger.error(f"Failed to get a valid response from {url}.")
    return {}

这种方式绕过requests自动编码检测，强制使用UTF-8解码，确保特殊字符（如长破折号—）正确解析。

3. 配置Azure函数App的全局UTF-8环境 #

Azure函数App的默认系统编码可能不是UTF-8，可添加环境变量强制Python使用UTF-8：
在Azure函数App的配置页面中添加：

• 键：PYTHONUTF8
• 值：1
  此配置会强制Python以UTF-8处理所有文本操作，从根源上避免编码问题。

验证方法 #

修改后，打印解析后的magic_set_data['data']['cards'][0]['type']，确认显示为Legendary Planeswalker — Jace，与Django环境一致后，执行数据库插入操作即可正常完成。

内容的提问来源于stack exchange，提问作者Ross