如何用Python提取JSON行格式字段数据并解决分词问题

问题背景 #

需要从JSON行格式的Amazon评论文件中提取summary和reviewText字段，分别存入review.txt和summary.txt用于后续分词分析。原代码运行后出现两个问题：

1. summary内容无分隔符，全部粘连在一起，示例如下：

A little bit boring for meExcellent Grown Up TVWay too boring for meRobson Green is mesmerizing

2. 代码在循环内提前关闭文件，导致后续循环无法写入。

用户提供的JSON行示例：

{"reviewerID": "A11N155CW1UV02", "asin": "B000H00VBQ", "reviewerName": "AdrianaM", "helpful": [0, 0], "reviewText": "I had big expectations because I love English TV, in particular Investigative and detective stuff but this guy is really boring. It didn't appeal to me at all.", "overall": 2.0, "summary": "A little bit boring for me", "unixReviewTime": 1399075200, "reviewTime": "05 3, 2014"}
{"reviewerID": "A3BC8O2KCL29V2", "asin": "B000H00VBQ", "reviewerName": "Carol T", "helpful": [0, 0], "reviewText": "I highly recommend this series. It is a must for anyone who is yearning to watch \"grown up\" television. Complex characters and plots to keep one totally involved. Thank you Amazin Prime.", "overall": 5.0, "summary": "Excellent Grown Up TV", "unixReviewTime": 1346630400, "reviewTime": "09 3, 2012"}
{"reviewerID": "A60D5HQFOTSOM", "asin": "B000H00VBQ", "reviewerName": "Daniel Cooper \"dancoopermedia\"", "helpful": [0, 1], "reviewText": "This one is a real snoozer. Don't believe anything you read or hear, it's awful. I had no idea what the title means. Neither will you.", "overall": 1.0, "summary": "Way too boring for me", "unixReviewTime": 1381881600, "reviewTime": "10 16, 2013"}

原错误代码：

import json
rt = open("review.txt", "a") #creates new file for storage
su = open("summary.txt", "a")


with open("/Users/anano/Desktop/MAXWELL/SPRING/NLP/Amazon_Instant_Video_5.json") as json_file:
    for line in json_file: #runs the loop to extract info
        data = json.loads(line)
        rt.write(data['reviewText'])
        su.write(data['summary'])
        rt.close()
        su.closed()

错误分析 #

• 文件关闭错误：在循环内部调用rt.close()和su.close()（原代码误写为su.closed()），导致第一次循环后文件就被关闭，后续循环无法再写入数据，直接报错。
• 内容粘连问题：写入字段内容时未添加分隔符（如换行符），所有summary和reviewText内容直接拼接在一起，无法区分单条内容，影响后续分词。

修正后的代码 #

import json

# 使用with语句管理所有文件，自动处理打开/关闭，避免手动操作错误
with open("/Users/anano/Desktop/MAXWELL/SPRING/NLP/Amazon_Instant_Video_5.json") as json_file, \
     open("review.txt", "w") as rt, \
     open("summary.txt", "w") as su:
    for line in json_file:
        data = json.loads(line)
        # 写入时添加换行符，确保每条内容单独一行
        rt.write(data['reviewText'] + '\n')
        su.write(data['summary'] + '\n')

改进说明 #

1. 文件管理优化：用with语句同时管理输入和输出文件，代码块结束后自动关闭所有文件，彻底避免手动关闭的错误。
2. 添加分隔符：写入每个字段时追加\n换行符，让每条summary和reviewText单独占一行，解决内容粘连问题，方便后续分词处理。
3. 文件模式调整：将输出文件模式从a（追加）改为w（写入），如果文件已存在会清空后写入，避免重复运行时内容重复；若需要保留历史内容，可改回a。

效果示例 #

修正后summary.txt的内容会是：

A little bit boring for me
Excellent Grown Up TV
Way too boring for me

内容的提问来源于stack exchange，提问作者Anano Mikadze