如何按timestamp字段排序JSONL文件？附问题排查与解决

原始JSONL日志数据 #

{"log_level": "DEBUG", "timestamp": "2022-12-04 18:04:09", "message": "Fry saw a man on the Mars yesterday"}
{"log_level": "INFO", "timestamp": "2022-12-03 11:21:35", "message": "Bender played a mall near the Square Garden today"}
{"log_level": "ERROR", "timestamp": "2022-12-03 11:21:42", "message": "Dr. Zoidberg took a mall at park day after tomorrow"}
{"log_level": "DEBUG", "timestamp": "2022-12-03 11:21:49", "message": "Fry built a fish at park today"}
{"log_level": "WARNING", "timestamp": "2022-12-03 11:21:55", "message": "Dr. Zoidberg brought a boat at park tomorrow"}
{"log_level": "ERROR", "timestamp": "2022-12-03 11:21:57", "message": "Farnsworth killed an apple near the Square Garden today"}

初始错误代码及报错 #

错误代码 #

import collections
import jsonlines
from datetime import datetime

def sort_merged_files(merged_file):

    with open(merged_file) as writer:
        dict = collections.defaultdict(list)
        for obj in jsonlines.Reader(writer):
            for k1, v1 in obj.items():
                dict[k1].append(v1)

        sorted_date = sorted(
            dict, key=lambda x: datetime.strptime(x["timestamp"], "%Y-%m-%d")
        )
        print(sorted_date)

报错信息 #

TypeError: string indices must be integers

错误原因 #

这段代码的问题在于，你把每个日志对象的键值对拆分存入了defaultdict，导致sorted()函数迭代的是字典的键（字符串类型），而不是完整的日志对象。所以lambda表达式里的x是字符串，尝试用x["timestamp"]访问就会触发类型错误——字符串只能用整数索引。

修正后的正确代码 #

import jsonlines
import time
import json

def sort_merged_files(merged_file):
    with open(merged_file) as reader:
        list_log = []
        # 将每个日志对象存入列表
        for obj in jsonlines.Reader(reader):
            list_log.append(obj)
        # 按完整时间格式解析timestamp后排序
        sorted_list = sorted(
            list_log, key=lambda x: time.mktime(time.strptime(x["timestamp"], "%Y-%m-%d %H:%M:%S"))
        )
        # 将排序后的日志写回JSONL文件
        with open(merged_file, "w") as f:
            for dic in sorted_list:
                json.dump(dic, f)
                f.write("\n")
        del sorted_list, list_log

修正说明 #

1. 直接保留完整的日志对象并存入列表，避免拆分键值对破坏数据结构
2. 使用包含时分秒的完整时间格式%Y-%m-%d %H:%M:%S解析timestamp，保证排序精度
3. 将排序后的日志重新写入原文件，保持JSONL的每行一个对象格式

内容的提问来源于stack exchange，提问作者ChungusNAPAS