Python2.7升级至3.9（配pandas1.1.5）遇_NoValueType类型错误的解决

• API请求未携带Headers：原代码定义了headers但未传入requests.get，可能被Yahoo Finance的反爬机制拦截，返回异常JSON结构，其中包含_NoValueType特殊空值。
• 版本兼容性差异：Python2.7环境下的旧版requests/pandas会自动将_NoValueType转换为兼容的空值类型（如NaN），但Python3.9搭配pandas1.1.5时对类型校验更严格，无法直接将该类型转换为数值类型，导致创建DataFrame时报错。
• 返回数据存在特殊空值：Yahoo Finance API返回的部分字段值为requests库定义的_NoValue类型，pandas无法直接解析该类型为有效数据类型。

步骤1：补全API请求的Headers #

确保请求携带User-Agent，避免被反爬返回异常数据：

query = requests.get('https://query1.finance.yahoo.com/v7/finance/quote?symbols=AALI.JK', headers=headers)

步骤2：预处理JSON数据，替换_NoValueType为None #

递归遍历返回的JSON数据，将_NoValue替换为None（pandas会自动将None转换为NaN，兼容数值类型处理）：

from requests.models import _NoValue

def replace_novalue(obj):
    if isinstance(obj, dict):
        return {k: replace_novalue(v) for k, v in obj.items()}
    elif isinstance(obj, list):
        return [replace_novalue(item) for item in obj]
    elif obj is _NoValue:
        return None
    else:
        return obj

# 处理返回的result数据
result_data = replace_novalue(data['quoteResponse']['result'])
data = pd.DataFrame(result_data)

步骤3：增强代码鲁棒性（可选） #

添加字段存在性校验，避免因API返回字段缺失导致后续报错：

# 处理时间字段时添加错误处理
if 'regularMarketTime' in data.columns:
    data['regularMarketTime'] = pd.to_datetime(data['regularMarketTime'], unit='s', errors='coerce').dt.strftime("%Y-%m-%d")

# 过滤存在的列，避免KeyError
selected_cols = ['regularMarketTime','symbol','regularMarketOpen','regularMarketDayHigh','regularMarketDayLow','regularMarketPrice','regularMarketVolume']
existing_cols = [col for col in selected_cols if col in data.columns]
data = data[existing_cols]

# 初始化data_append（原代码未初始化会报错）
if 'data_append' not in locals():
    data_append = pd.DataFrame()
data_append = data_append.append(data)

完整修复后代码 #

import requests
import pandas as pd
from requests.models import _NoValue

agent = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36'
headers = {'User-Agent': agent}
# 传入Headers请求数据
query = requests.get('https://query1.finance.yahoo.com/v7/finance/quote?symbols=AALI.JK', headers=headers)
data = query.json()

# 替换_NoValue为None
def replace_novalue(obj):
    if isinstance(obj, dict):
        return {k: replace_novalue(v) for k, v in obj.items()}
    elif isinstance(obj, list):
        return [replace_novalue(item) for item in obj]
    elif obj is _NoValue:
        return None
    else:
        return obj

result_data = replace_novalue(data['quoteResponse']['result'])
data = pd.DataFrame(result_data)

# 处理时间字段
if 'regularMarketTime' in data.columns:
    data['regularMarketTime'] = pd.to_datetime(data['regularMarketTime'], unit='s', errors='coerce').dt.strftime("%Y-%m-%d")

# 选择存在的列
selected_cols = ['regularMarketTime','symbol','regularMarketOpen','regularMarketDayHigh','regularMarketDayLow','regularMarketPrice','regularMarketVolume']
existing_cols = [col for col in selected_cols if col in data.columns]
data = data[existing_cols]

# 初始化并追加数据
if 'data_append' not in locals():
    data_append = pd.DataFrame()
data_append = data_append.append(data)

内容的提问来源于stack exchange，提问作者bkcollection