读取截断JSON文件至Pandas DataFrame报错，寻求解决方法

我有一个截断的JSON文件，尝试用Pandas的read_json方法读取到DataFrame时失败，代码如下：

import pandas as pd

df = pd.read_json("C:/Users/user1/Documents/history_truncated.json")
df.head()

运行后报错：

ValueError: Unexpected character found when decoding array value (2)

试过使用绝对路径的方案，但无效。以下是JSON文件的末尾几行内容：

"departurePlatformName":null,"departureBoardingActivity":0,"departureBoardingActivitySpecified":true,"departureStopAssignment":null,
"departureOperatorRefs":null,"aimedHeadwayInterval":null,"expectedHeadwayInterval":null,"distanceFromStop":null,"numberOfStopsAway":null,"extensions":null},{"stopPointRef":{"value":"SE:276:Quay:9022012013003041"},"visitNumber":"1","order":"7","stopPointName":null,"item":false,"itemElementName":0,"predictionInaccurateSpecified":false,"occupancySpecified":false,"timingPointSpecified":false,"boardingStretch":false,"requestStop":false,"originDisplay":null,"destinationDisplay":null,"callNote":null,"facilityConditionElement":null,"facilityChangeElement":null,"situationRef":null,"aimedArrivalTime":"2022-08-30T09:33:00+02:00","aimedArrivalTimeSpecified":true,"expectedArrivalTime":"2022-08-30T09:33:00+02:00","expectedArrivalTimeSpecified":true,"expectedArrivalPredictionQuality":null,"arrivalStatus":0,"arrivalStatusSpecified":true,"arrivalProximityText":null,"arrivalPlatformName":null,"arrivalBoardingActivity":0,"arrivalBoardingActivitySpecified":true,"arrivalStopAssignment":null,"arrivalOperatorRefs":null,"aimedDepartureTimeSpecified":false,"expectedDepartureTimeSpecified":false,"provisionalExpectedDepartureTimeSpecified":false,"earliestExpectedDepartureTimeSpecified":false,"expectedDeparturePredictionQuality":null,"aimedLatestPassengerAccessTimeSpecified":false,"expectedLatestPassengerAccessTimeSpecified":false,"departureStatus":0,"departureStatusSpecified":true,"departureProximityText":null,"departurePlatformName":null,"departureBoardingActivity":0,
"departureBoardingActivitySpecified":true,
"departureStopAssignment":null,
"departureOperatorRefs":null,"aimedHeadwayInterval":null,
"expectedHeadwayInterval":null,"distanceFromStop":null,
"numberOfStopsAway":null,"extensions":null}],"isCompleteStopSequence":true,
"extensions":null}
}]
}]]}
}

1. 手动修复JSON语法错误 #

从提供的末尾内容看，JSON存在结构截断问题（嵌套的{}/[]未正确闭合）。可以：

• 用支持JSON校验的编辑器（比如VS Code）打开文件，编辑器会高亮语法错误位置，手动补全缺失的闭合括号或引号。
• 统计文件中{和}、[和]的数量差值，对应补全缺失的闭合符号。

2. 用代码自动尝试修复并读取 #

通过代码统计括号数量，补全缺失的闭合符号后再解析：

import json
import pandas as pd

def fix_and_load_json(file_path):
    with open(file_path, 'r', encoding='utf-8') as f:
        content = f.read()
    
    # 统计开闭括号数量，补全缺失的闭合符号
    open_braces = content.count('{')
    close_braces = content.count('}')
    open_brackets = content.count('[')
    close_brackets = content.count(']')
    
    content += '}' * (open_braces - close_braces)
    content += ']' * (open_brackets - close_brackets)
    
    try:
        data = json.loads(content)
        # 根据JSON结构调整DataFrame的生成方式，比如嵌套结构可能需要用json_normalize
        return pd.json_normalize(data)
    except json.JSONDecodeError as e:
        print(f"修复后仍有语法错误: {e}")
        return None

df = fix_and_load_json("C:/Users/user1/Documents/history_truncated.json")
if df is not None:
    print(df.head())

3. 尝试分块读取解析 #

如果文件过大，截断位置不明确，可以逐行拼接内容，找到能正常解析的最大有效部分：

import json
import pandas as pd

file_path = "C:/Users/user1/Documents/history_truncated.json"
with open(file_path, 'r', encoding='utf-8') as f:
    lines = f.readlines()

# 从第一行开始逐步拼接，直到解析成功
for line_count in range(1, len(lines)+1):
    try:
        partial_content = ''.join(lines[:line_count])
        data = json.loads(partial_content)
        df = pd.json_normalize(data)
        print(f"成功解析前{line_count}行数据")
        print(df.head())
        break
    except json.JSONDecodeError:
        continue

4. 调整read_json参数尝试 #

如果JSON是行分隔格式（每行一个独立JSON对象），可以添加lines=True参数尝试读取：

df = pd.read_json("C:/Users/user1/Documents/history_truncated.json", lines=True)

内容的提问来源于stack exchange，提问作者OLGJ