如何用Pandas对工作站日志Execution区间内的S/N值排序？

需求说明 #

将日志中每个Execution started至Execution ended区间内的VAR记录，按S/N（格式如W2300900009）从小到大排序，非区间内的日志行保持原有顺序。

原始日志 #

<!--00:00:13:23 - Execution started - 01042141053 - B Side
00:02:59:10 - VAR 10.2 = 'W2300900009'
00:02:50:78 - VAR 8.2 = 'W2300900007' 
00:02:42:51 - VAR 6.2 = 'W2300900005' 
00:03:45:18 - Execution ended'
System in Power Counter = 49035:41:56 
'00:04:01:29 - Execution started - 01042141053 
'00:04:40:28 - VAR 4.2 = 'W2300900023'
'00:04:36:36 - VAR 3.2 = 'W2300900022'
'00:04:32:34 - VAR 2.2 = 'W2300900021'
'00:05:50:62 - Execution ended'

期望输出 #

'00:00:13:23 - Execution started - 01042141053 - B Side'
'00:02:42:51 - VAR 6.2 = 'W2300900005'
'00:02:50:78 - VAR 8.2 = 'W2300900007'
'00:02:59:10 - VAR 10.2 = 'W2300900009'
'00:03:45:18 - Execution ended'
System in Power Counter = 49035:41:56
'00:04:01:29 - Execution started - 01042141053
'00:04:32:34 - VAR 2.2 = 'W2300900021'   
'00:04:36:36 - VAR 3.2 = 'W2300900022'
'00:04:40:28 - VAR 4.2 = 'W2300900023'
'00:05:50:62 - Execution ended'-->

实现步骤与代码 #

1. 读取日志内容：将日志转换为Pandas Series，每行作为单独元素。
2. 标记区间分组：为每个Execution区间分配唯一ID，非区间行用独立标识区分。
3. 提取S/N字段：针对VAR行用正则匹配提取S/N值，用于排序依据。
4. 分组排序处理：对每个Execution区间内的VAR记录按S/N排序，非区间行保持原顺序。
5. 输出处理后日志：将处理后的行重新组合为完整日志文本。

完整Python代码如下：

import pandas as pd
import re

# 日志内容（如果是文件，可替换为pd.read_csv('log.txt', header=None, names=['line'])）
log_content = """<!--00:00:13:23 - Execution started - 01042141053 - B Side
00:02:59:10 - VAR 10.2 = 'W2300900009'
00:02:50:78 - VAR 8.2 = 'W2300900007' 
00:02:42:51 - VAR 6.2 = 'W2300900005' 
00:03:45:18 - Execution ended'
System in Power Counter = 49035:41:56 
'00:04:01:29 - Execution started - 01042141053 
'00:04:40:28 - VAR 4.2 = 'W2300900023'
'00:04:36:36 - VAR 3.2 = 'W2300900022'
'00:04:32:34 - VAR 2.2 = 'W2300900021'
'00:05:50:62 - Execution ended'"""

# 转换为Series格式
df = pd.Series(log_content.split('\n'), name='line')

# 为每行标记所属分组ID
group_id = 0
in_execution = False
groups = []
for line in df:
    if 'Execution started' in line:
        in_execution = True
        groups.append(group_id)
    elif 'Execution ended' in line:
        in_execution = False
        groups.append(group_id)
        group_id += 1
    else:
        groups.append(group_id if in_execution else -(len(groups)+1))

df['group'] = groups

# 提取VAR行的S/N值
df['sn'] = df['line'].apply(lambda x: re.search(r"'(W\d+)'", x).group(1) if 'VAR' in x else None)

# 分组处理排序逻辑
processed_lines = []
for g in df['group'].unique():
    group_df = df[df['group'] == g].copy()
    if g >= 0:
        # 拆分区间内的启动行、VAR行、结束行，对VAR行按S/N排序
        started_line = group_df[group_df['line'].str.contains('Execution started')]
        var_lines = group_df[group_df['line'].str.contains('VAR')].sort_values('sn')
        ended_line = group_df[group_df['line'].str.contains('Execution ended')]
        processed_group = pd.concat([started_line, var_lines, ended_line])
    else:
        processed_group = group_df
    processed_lines.extend(processed_group['line'].tolist())

# 输出最终日志
final_log = '\n'.join(processed_lines)
print(final_log)

代码说明 #

• 分组标记：通过遍历识别Execution区间的起止，为每个区间分配正ID，非区间行用负ID，确保非区间内容顺序不受影响。
• S/N提取：用正则表达式精准匹配VAR行中的S/N字符串，保证排序依据的准确性。
• 排序逻辑：每个Execution区间内，优先保留启动行和结束行的位置，仅对中间的VAR行按S/N升序排列，兼顾日志结构完整性和排序需求。

内容的提问来源于stack exchange，提问作者František Veselý