Python提取日志文件新增差异行：代码未达预期求解决

我编写了Python程序用于对比备份日志文件（backup_file）与当前日志文件（curr_file），初始两者内容一致。当前文件会更新记录，我希望以类似集合差集的形式提取当前文件中的新增行，但未得到预期结果。相关代码、文件内容及预期输出如下：

原代码 #

import filecmp
import shutil

if __name__ == '__main__':

    backup_file ="D:\\PythonScripts\\SIC_Automation\\Backup.log" #backup_file for comparison with if the current file have any updated entries
    curr_file= "D:\\PythonScripts\\SIC_Automation\\Mock_files\\FailedTid.log" #failed Transaction file from SIC


    with open(backup_file) as f1:
        backup_file_lines = set(f1.readlines())
    
    with open(curr_file) as f2:
        curr_file_lines = set(f2.readlines())

    if (not(filecmp.cmp(backup_file, curr_file))): 
        #1 get the diffence from the file and store in a variable
        diff_error_line = curr_file_lines - backup_file_lines

        print(diff_error_line) #Print the list of difference in logs- TEMP CODE
    else:
        print("Both files are identical")

备份日志文件内容 #

2023-03-17 19:30:03,699 - Tid: AC19DD1D84856414D1F60001   Error: Input tax has been specified but there are no taxes to which it may be distributed. Please verify the details for this transaction.
2023-03-17 19:50:18,305 - Tid: AC19DD1D84856414D1F60001   Error: Input tax has been specified but there are no taxes to which it may be distributed. Please verify the details for this transaction.
2023-03-17 20:10:28,767 - Tid: AC19DD1D84856414D1F60001   Error: Input tax has been specified but there are no taxes to which it may be distributed. Please verify the details for this transaction.

当前日志文件内容 #

2023-03-17 19:30:03,699 - Tid: AC19DD1D84856414D1F60001   Error: Input tax has been specified but there are no taxes to which it may be distributed. Please verify the details for this transaction.
2023-03-17 19:50:18,305 - Tid: AC19DD1D84856414D1F60001   Error: Input tax has been specified but there are no taxes to which it may be distributed. Please verify the details for this transaction.
2023-03-17 20:10:28,767 - Tid: AC19DD1D84856414D1F60001   Error: Input tax has been specified but there are no taxes to which it may be distributed. Please verify the details for this transaction.
2023-03-17 20:30:40,105 - Tid: AC19DD1D84856414D1F60001   Error: Input tax has been specified but there are no taxes to which it may be distributed. Please verify the details for this transaction.

预期输出 #

2023-03-17 20:30:40,105 - Tid: AC19DD1D84856414D1F60001   Error: Input tax has been specified but there are no taxes to which it may be distributed. Please verify the details for this transaction.

1. 集合格式与顺序问题：set(f.readlines())将行转为集合元素，集合是无序的，打印时会输出带{}的集合格式，而非预期的单行文本。
2. 冗余的文件对比：filecmp.cmp会对整个文件做对比，而代码已经通过集合做了差集计算，这个判断属于重复操作。
3. 换行符干扰：readlines()会保留每行末尾的换行符（如\n或\r\n），若两个文件换行符格式不一致，会导致相同内容的行被判定为不同。
4. 重复行处理问题：集合会自动去重，若备份文件或当前文件存在重复行，差集计算会丢失正确的新增行。

方案1：针对追加式日志（高效推荐） #

大多数日志文件是追加写入的，新增行只会出现在文件末尾。这种情况下可以直接读取备份文件的行数，再读取当前文件中超出该行数的内容，效率更高：

if __name__ == '__main__':
    backup_file = "D:\\PythonScripts\\SIC_Automation\\Backup.log"
    curr_file = "D:\\PythonScripts\\SIC_Automation\\Mock_files\\FailedTid.log"

    # 获取备份文件的总行数
    with open(backup_file, 'r') as f:
        backup_line_count = sum(1 for _ in f)
    
    # 读取当前文件中新增的行（跳过备份已有的行数）
    with open(curr_file, 'r') as f:
        # 跳过前backup_line_count行
        for _ in range(backup_line_count):
            next(f)
        # 收集并处理新增行（去除换行符）
        new_lines = [line.rstrip('\n') for line in f]
    
    if new_lines:
        for line in new_lines:
            print(line)
        # 可选：更新备份文件，同步当前文件内容
        import shutil
        shutil.copy(curr_file, backup_file)
    else:
        print("Both files are identical")

方案2：通用场景（支持任意位置新增行） #

如果新增行可能出现在文件任意位置，可采用此方案，准确提取所有不在备份中的行并保留原顺序：

if __name__ == '__main__':
    backup_file = "D:\\PythonScripts\\SIC_Automation\\Backup.log"
    curr_file = "D:\\PythonScripts\\SIC_Automation\\Mock_files\\FailedTid.log"

    # 读取文件内容，去除换行符避免格式干扰
    with open(backup_file, 'r') as f1:
        backup_lines = {line.rstrip('\n') for line in f1}
    
    with open(curr_file, 'r') as f2:
        curr_lines = [line.rstrip('\n') for line in f2]
    
    # 提取当前文件中不在备份里的行，保留原顺序
    new_lines = [line for line in curr_lines if line not in backup_lines]
    
    if new_lines:
        for line in new_lines:
            print(line)
        # 可选：更新备份文件，写入当前文件所有内容
        with open(backup_file, 'w') as f:
            f.write('\n'.join(curr_lines) + '\n')
    else:
        print("Both files are identical")

内容的提问来源于stack exchange，提问作者Honey Chintal