如何从复杂CEF日志字符串中提取AV动作与恶意软件名称

需求说明 #

需从CEF格式日志中提取两类信息：

• AV:标识后的杀毒动作（如File quarantined）
• 该动作后由|分隔的恶意软件名称（如Trojan.Win64.SHELMA.SMB1）

失败尝试 #

之前使用的正则表达式及代码无法正确提取目标内容：

import re

pattern = '(AV:)(?s)(.*?)\|'
a = [re.search(pattern,m) for m in malware]
print(a)

解决方案 #

正确正则表达式 #

使用以下正则可以精准匹配目标内容：

AV:([^|]+)\|([^|]+)

• AV: 匹配固定前缀
• ([^|]+) 第一个捕获组，匹配AV:后到第一个|之间的所有非|字符，即杀毒动作
• \| 匹配分隔用的竖线
• ([^|]+) 第二个捕获组，匹配竖线后到下一个|之间的所有非|字符，即恶意软件名称

完整代码示例 #

import re

malware = [
    'Mar 07 2023 17:15:00 abcd.manage.trendmicro.com CEF:0|Trend Micro|Apex Central|2019|AV:File quarantined|Trojan.Win64.SHELMA.SMB1|3|deviceExternalId=313 rt=2022-12-21 08:44:17 cnt=1 dhost=NB-SUPPORT TMCMLogDetectedHost=NB-SUPPORT duser=ACME\\john.smith act=File quarantined cn1Label=Pattern cn1=1814300 cn2Label=Second_Action cn2=1 cs1Label=VLF_FunctionCode cs1=Real-time Scan cs2Label=Engine cs2=22.580.1004 cs3Label=Product_Version cs3=14.0 cs4Label=CLF_ReasonCode cs4=virus log cs5Label=First_Action_Result cs5=File quarantined cs6Label=Second_Action_Result cs6=N/A cat=1703 dvchost=cpnlug.manage.trendmicro.com cn3Label=Overall_Risk_Rating cn3=0 fname=66e9f4d4-df39-488d-8cf8-bdcf5d890598.tmp filePath=C:\\Users\\emil\\Downloads\\ msg=NONAMEFL dst=10.18.13.90 TMCMLogDetectedIP=10.18.13.90 fileHash=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX deviceFacility=Apex One ApexCentralHost=Apex Central as a Service devicePayloadId=xxx-xxxxx-xxx-xxx TMCMdevicePlatform=Windows 10 10.0 (Build 19044) deviceNtDomain=N/A dntdom=Client\\Notebook\\ ',
    'Mar 07 2023 17:15:00 abcd.manage.trendmicro.com CEF:0|Trend Micro|Apex Central|2019|AV:File passed|Trojan.Win64.SHELMA.SMB1|3|deviceExternalId=314 rt=2022-12-21 08:45:17 cnt=1 dhost=NB-SUPPORT TMCMLogDetectedHost=NB-SUPPORT duser=ACME\\john.smith act=File quarantined cn1Label=Pattern cn1=1814300 cn2Label=Second_Action cn2=1 cs1Label=VLF_FunctionCode cs1=Real-time Scan cs2Label=Engine cs2=22.580.1004 cs3Label=Product_Version cs3=14.0 cs4Label=CLF_ReasonCode cs4=virus log cs5Label=First_Action_Result cs5=File quarantined cs6Label=Second_Action_Result cs6=N/A cat=1703 dvchost=cpnlug.manage.trendmicro.com cn3Label=Overall_Risk_Rating cn3=0 fname=rev_shell.exe filePath=C:\\Users\\emil\\Downloads\\ msg=NONAMEFL dst=10.18.13.90 TMCMLogDetectedIP=10.18.13.90 fileHash=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX deviceFacility=Apex One ApexCentralHost=Apex Central as a Service devicePayloadId=xxx-xxxxx-xxx-xxx TMCMdevicePlatform=Windows 10 10.0 (Build 19044) deviceNtDomain=N/A dntdom=Client\\Notebook\\ ',
    'Mar 07 2023 17:15:00 abcd.manage.trendmicro.com CEF:0|Trend Micro|Apex Central|2019|AV:File cleaned|TROJ_GEN.R002C0DKG22|3|deviceExternalId=315 rt=2022-12-21 10:20:31 cnt=1 dhost=NB-SUPPORT TMCMLogDetectedHost=NB-SUPPORT duser=ACME\\john.smit act=File cleaned cn1Label=Pattern cn1=1814500 cn2Label=Second_Action cn2=1 cs1Label=VLF_FunctionCode cs1=Real-time Scan cs2Label=Engine cs2=22.580.1004 cs3Label=Product_Version cs3=14.0 cs4Label=CLF_ReasonCode cs4=virus log cs5Label=First_Action_Result cs5=File cleaned cs6Label=Second_Action_Result cs6=N/A cat=1703 dvchost=cpnlug.manage.trendmicro.com cn3Label=Overall_Risk_Rating cn3=1 fname=aowect.dll filePath=C:\\Users\\emil\\AppData\\Local\\Temp\\ dst=10.18.13.90 TMCMLogDetectedIP=10.18.13.90 fileHash=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX deviceFacility=Apex One ApexCentralHost=Apex Central as a Service devicePayloadId=xxx-xxxxx-xxx-xxx TMCMdevicePlatform=Windows 10 10.0 (Build 19044) deviceNtDomain=N/A dntdom=Client\\Notebook\\ ',
    'Mar 07 2023 17:15:00 abcd.manage.trendmicro.com CEF:0|Trend Micro|Apex Central|2019|AV:Unable to upload file|TSC_GENCLEAN|3|deviceExternalId=316 rt=2022-12-21 13:37:42 cnt=1 dhost=NB-SUPPORT TMCMLogDetectedHost=NB-SUPPORT duser=ACME\\john.smit act=File cleaned cn1Label=Pattern cn1=1632 cn2Label=Second_Action cn2=1 cs1Label=VLF_FunctionCode cs1=Damage Cleanup Services cs2Label=Engine cs2=7.5.1184 cs3Label=Product_Version cs3=14.0 cs4Label=CLF_ReasonCode cs4=virus log cs5Label=First_Action_Result cs5=File cleaned cs6Label=Second_Action_Result cs6=N/A cat=1703 dvchost=cpnlug.manage.trendmicro.com cn3Label=Overall_Risk_Rating cn3=0 fname=Non confermato 184296.crdownload filePath=C:\\Users\\emil\\Downloads\\ dst=10.18.13.90 TMCMLogDetectedIP=10.18.13.90 deviceFacility=Apex One ApexCentralHost=Apex Central as a Service devicePayloadId=xxx-xxxxx-xxx-xxx TMCMdevicePlatform=Windows 10 10.0 (Build 19044) deviceNtDomain=N/A dntdom=Client\\Notebook\\ ',
    'Mar 07 2023 17:15:00 abcd.manage.trendmicro.com CEF:0|Trend Micro|Apex Central|2019|AV:File quarantinedd|Troj.Win32.TRX.XXPE50FFF063|3|deviceExternalId=317 rt=2022-12-21 13:37:49 cnt=1 dhost=NB-SUPPORT TMCMLogDetectedHost=NB-SUPPORT duser=ACME\\john.smit act=File cleaned cn1Label=Pattern cn1=1632 cn2Label=Second_Action cn2=1 cs1Label=VLF_FunctionCode cs1=Damage Cleanup Services cs2Label=Engine cs2=7.5.1184 cs3Label=Product_Version cs3=14.0 cs4Label=CLF_ReasonCode cs4=virus log cs5Label=First_Action_Result cs5=File cleaned cs6Label=Second_Action_Result cs6=N/A cat=1703 dvchost=cpnlug.manage.trendmicro.com cn3Label=Overall_Risk_Rating cn3=0 fname=pumpkin-2.7.3.exe filePath=C:\\Users\\emil\\Downloads\\ dst=10.18.13.90 TMCMLogDetectedIP=10.18.13.90 deviceFacility=Apex One ApexCentralHost=Apex Central as a Service devicePayloadId=xxx-xxxxx-xxx-xxx TMCMdevicePlatform=Windows 10 10.0 (Build 19044) deviceNtDomain=N/A dntdom=Client\\Notebook\\ '
]

pattern = r'AV:([^|]+)\|([^|]+)'
results = []
for log in malware:
    match = re.search(pattern, log)
    if match:
        action = match.group(1)
        malware_name = match.group(2)
        results.append({"杀毒动作": action, "恶意软件名称": malware_name})

# 输出提取结果
for item in results:
    print(f"杀毒动作: {item['杀毒动作']}, 恶意软件名称: {item['恶意软件名称']}")

输出结果 #

杀毒动作: File quarantined, 恶意软件名称: Trojan.Win64.SHELMA.SMB1
杀毒动作: File passed, 恶意软件名称: Trojan.Win64.SHELMA.SMB1
杀毒动作: File cleaned, 恶意软件名称: TROJ_GEN.R002C0DKG22
杀毒动作: Unable to upload file, 恶意软件名称: TSC_GENCLEAN
杀毒动作: File quarantinedd, 恶意软件名称: Troj.Win32.TRX.XXPE50FFF063

内容的提问来源于stack exchange，提问作者CUI