如何筛选Python共享日志中仅来自特定文件的日志行

核心优化思路 #

不需要引入额外依赖，只要对现有逻辑做少量调整就能得到灵活易维护的实现，比仅添加__init__参数的方案更简洁：

• 把硬编码的过滤关键词、日志路径、日期前缀全部抽成可配置参数，无需修改核心逻辑就能调整过滤规则
• 用集合存储黑白名单搭配any()生成器表达式，替代冗长的or条件判断，新增过滤关键词无需修改判断逻辑
• 同时支持黑名单、白名单两种过滤模式，适配不同的筛选场景
• 若不需要额外类方法，可直接用生成器函数实现，比类结构更轻量

轻量实现（生成器版本，推荐） #

import os

def filter_shared_log(log_name=None, blacklist=None, whitelist=None, date_prefix="2021-08"):
    # 缺省参数配置
    log_name = log_name or os.path.basename(__file__)
    blacklist = blacklist or {"hpack", "selenium", "urllib", "statsapi"}

    with open(f'/tmp/{log_name}.shared.log', 'r') as raw:
        skip = False
        for line in raw:
            if line.startswith(date_prefix):
                skip = False
            # 优先走白名单逻辑，未配置白名单则走黑名单逻辑
            if whitelist is not None:
                skip = not any(keyword in line for keyword in whitelist)
            else:
                if any(keyword in line for keyword in blacklist):
                    skip = True
            if not skip:
                yield line

使用示例 #

# 默认使用内置黑名单过滤
for log_line in filter_shared_log():
    print(log_line)

# 自定义白名单，仅保留包含指定关键词的日志
for log_line in filter_shared_log(whitelist={"my_business", "order_module"}):
    print(log_line)

# 自定义日志名称、过滤日期前缀
for log_line in filter_shared_log(log_name="my_project", date_prefix="2024-05"):
    print(log_line)

类结构版本（需要扩展类方法时使用） #

import os
class FocusedLog:
    ''' Iterate through only the parts of the log we care about. '''
    def __init__(self, log_name=None, blacklist=None, whitelist=None, date_prefix="2021-08"):
        self.log_name = log_name or os.path.basename(__file__)
        self.blacklist = blacklist or {"hpack", "selenium", "urllib", "statsapi"}
        self.whitelist = whitelist
        self.date_prefix = date_prefix

    def __iter__(self):
        with open(f'/tmp/{self.log_name}.shared.log', 'r') as raw:
            skip = False
            for line in raw:
                if line.startswith(self.date_prefix):
                    skip = False
                if self.whitelist is not None:
                    skip = not any(keyword in line for keyword in self.whitelist)
                else:
                    if any(keyword in line for keyword in self.blacklist):
                        skip = True
                if not skip:
                    yield line

内容的提问来源于stack exchange，提问作者Joshua Snider