Python日志提取指定行遇编码错误及输出冗余问题求助

我有一个log.txt文件，需要提取以"NEC Input Before Replace"和"NEC Matching Word"开头的行。最初编写的Python代码运行时出现UnicodeDecodeError，添加utf-8编码后解决了编码问题，但输出却包含目标行之后的所有无关内容，求原因及解决方法。

日志文件(log.txt)内容 #

2022-10-12 18:15:22.992 0026/? I/AsrDecActor25: channels=1, size=82434
2022-10-12 18:15:22.992 0026/? I/AsrDecActor25: waiting asr-core ready: 12 secs
2022-10-12 18:15:23.058 0199/? I/AsrDecActor27: asr state: START, true
2022-10-12 18:15:23.058 0199/? I/AsrDecActor27: asr state 2: START
2022-10-12 18:15:23.058 0199/? I/AsrDecActor27: end of decoding 57 true 0
NEC Input :secure folder app close it
NEC Replacement suggestion :Secure folder
NEC Input Before Replace : secure folder app close it
NEC Matching Word : secure folder app
Replaced Word  : Secure folder
NEC Output After Replace : Secure folder close it
Changes : 1
2022-10-12 18:15:23.060 0199/? I/LangPackActor: eASR [NEC] Run completed, Time: 2 ms
PostProcessSubstitutions::Output of question mark processing: secure folder uninstall Kare
[eITN] Input:Secure folder uninstall kare OutputSecure folder uninstall Kare
2022-10-12 18:15:23.069 0199/? I/LangPackActor: eASR [Timestamp] getTimestamp starts
2022-10-12 18:15:23.069 0199/? I/LangPackActor: eASR string2IntegerList 14 20 23 32 36 
2022-10-12 18:15:23.069 0199/? I/LangPackActor: eASR string2IntegerList 14 20 23 32 36 
2022-10-12 18:15:23.069 0199/? I/LangPackActor: eASR levenshteinMapping
2022-10-12 18:15:23.069 0199/? I/LangPackActor: eASR new ASRResult
2022-10-12 18:15:23.091 0021/? I/AsrDecActor26: decoding 

期望输出 #

NEC Input Before Replace : secure folder app close it
NEC Matching Word : secure folder app

初始代码及错误信息 #

初始代码：

#!/usr/bin/env python
f = open('log.txt')
f1 = open('output.txt', 'a')

doIHaveToCopyTheLine=False

for line in f.readlines():

    if 'NEC Input Before Replace' in line:
        doIHaveToCopyTheLine=True
    elif 'NEC Matching Word' in line:
        doIHaveToCopyTheLine=True

    if doIHaveToCopyTheLine:
        f1.write(line)

f1.close()
f.close()

错误信息：

UnicodeDecodeError                        Traceback (most recent call last)
Input In [3], in <cell line: 7>()
      3 f1 = open('output.txt', 'a')
      5 doIHaveToCopyTheLine=False
----> 7 for line in f.readlines():
      9     if 'NEC Input Before Replace' in line:
     10         doIHaveToCopyTheLine=True

File D:\Anaconda\lib\encodings\cp1252.py:23, in IncrementalDecoder.decode(self, input, final)
     22 def decode(self, input, final=False):
---&gt; 23     return codecs.charmap_decode(input,self.errors,decoding_table)[0]

UnicodeDecodeError: 'charmap' codec can't decode byte 0x8d in position 7524: character maps to <undefined>

修改后代码及异常输出 #

修改后代码（添加utf-8编码）：

#!/usr/bin/env python
f = open('log.txt','r',encoding='utf-8')
f1 = open('output.txt', 'a')

doIHaveToCopyTheLine=False

for line in f.readlines():

    if 'NEC Input Before Replace' in line:
        doIHaveToCopyTheLine=True
    elif 'NEC Matching Word' in line:
        doIHaveToCopyTheLine=True

    if doIHaveToCopyTheLine:
        f1.write(line)

f1.close()
f.close()

异常输出：包含目标行之后的所有无关内容。

原因分析 #

1. 编码问题原因：初始代码打开文件时未指定编码，系统默认使用cp1252编码读取utf-8格式的日志文件，导致无法解码部分特殊字节，触发UnicodeDecodeError。
2. 输出冗余内容原因：代码中的doIHaveToCopyTheLine变量被设为True后，没有任何逻辑将其重置为False。一旦匹配到第一个目标行，后续所有行都会被写入输出文件，因此包含了大量无关内容。

解决方法 #

方法一：直接匹配目标前缀（推荐）

不需要状态变量，直接判断每行是否以指定前缀开头，符合条件则写入输出文件。同时使用with语句自动管理文件，避免手动关闭的风险，且用w模式覆盖输出文件（避免追加旧内容）：

#!/usr/bin/env python
# 定义需要匹配的前缀
target_prefixes = ('NEC Input Before Replace', 'NEC Matching Word')

# with语句自动处理文件打开/关闭
with open('log.txt', 'r', encoding='utf-8') as input_file, open('output.txt', 'w', encoding='utf-8') as output_file:
    for line in input_file:
        # 检查当前行是否以目标前缀开头
        if line.startswith(target_prefixes):
            output_file.write(line)

方法二：修复状态变量逻辑

如果要保留原有的状态判断思路，需在匹配完所有目标行后重置状态：

#!/usr/bin/env python
with open('log.txt', 'r', encoding='utf-8') as input_file, open('output.txt', 'w', encoding='utf-8') as output_file:
    do_copy = False
    matched_lines = 0
    total_targets = 2  # 需要提取的目标行数
    
    for line in input_file:
        if 'NEC Input Before Replace' in line or 'NEC Matching Word' in line:
            do_copy = True
            matched_lines += 1
            # 匹配完所有目标行后关闭复制状态
            if matched_lines >= total_targets:
                do_copy = False
        
        if do_copy:
            output_file.write(line)

内容的提问来源于stack exchange，提问作者Turing101