如何在Python中生成每行一个JSON对象的JSONL文件？

问题说明 #

需要生成每行一个独立JSON对象的JSONL文件，格式示例：

{"file_name": "0001.png", "text": "This is a golden retriever playing with a ball"}
{"file_name": "0002.png", "text": "A german shepherd"}
{"file_name": "0003.png", "text": "One chihuahua"}

但当前代码生成的是包含所有对象的JSON数组，不符合要求。

原错误代码 #

import json
import pandas as pd

dt = pd.read_csv('data.csv')
df = pd.DataFrame(dt)

file_name = df['image']
file_caption = df['text']

data = []

for i in range(len(file_name)):
    entry = {"file_name": file_name[i], "text": file_caption[i]}
    data.append(entry)

json_object = json.dumps(data, indent=4)

# Writing to sample.json
with open("metadata.jsonl", "w") as outfile:
    outfile.write(json_object)

错误输出示例 #

[
    {
        "file_name": "images/image_0.jpg",
        "text": "Fattoush Salad with Roasted Potatoes"
    },
    {
        "file_name": "images/image_1.jpg",
        "text": "an analysis of self portrayal in novels by virginia woolf A room of one's own study guide contains a biography of virginia woolf, literature essays, quiz questions, major themes, characters, and a full summary and analysis about a room of one's own a room of one's own summary."
    },
    ...
]

修改方案 #

方法1：循环逐个序列化写入 #

无需把所有对象存入列表再整体序列化，直接在循环中逐个将单个JSON对象转为字符串，每行写入文件：

import json
import pandas as pd

df = pd.read_csv('data.csv')

with open("metadata.jsonl", "w") as outfile:
    for _, row in df.iterrows():
        entry = {"file_name": row['image'], "text": row['text']}
        # 序列化单个对象，不添加缩进保证单行格式
        json_str = json.dumps(entry)
        outfile.write(json_str + '\n')

方法2：使用Pandas内置to_json方法（更高效） #

Pandas原生支持生成JSONL格式，无需手动循环：

import pandas as pd

df = pd.read_csv('data.csv')
# 重命名列匹配目标字段，直接生成JSONL
df.rename(columns={'image': 'file_name'}, inplace=True)
df[['file_name', 'text']].to_json('metadata.jsonl', orient='records', lines=True)

关键说明 #

• 原代码错误在于用json.dumps(data)将整个列表序列化为JSON数组，而JSONL要求每行是独立的JSON对象，不能有数组的[]包裹。
• 序列化单个对象时不要加indent参数，避免生成多行格式的JSON，确保每行是紧凑的单个对象字符串。

内容的提问来源于stack exchange，提问作者mchd