如何用Bash脚本提取CSV中Severity列为空的行至新文件？

场景与需求 #

• 场景：拥有一个包含多列的CSV文件
• 需求：通过Bash脚本提取Severity列为空的行，并将完整行内容写入新的CSV文件

尝试的错误代码 #

awk -F -vpos_sev=${severity_column}, '!length(${vpos_sev})' ${source_log_file} | tee -a ${no_severity_logs_file}

变量说明 #

• severity_column=3 # 标记Severity所在的列位置
• source_log_file # 需要解析的CSV源文件路径
• no_severity_logs_file # 存储提取结果的目标文件路径

示例源文件 #

Header 1,Header 2,Severity,Header 4,Header 5
col1txt1,col2txt1,INFO,col4txt1,col5txt1
col1txt2,col2txt2,WARNING,col4txt2,col5txt2
col1txt3,col2txt3,ERROR,col4txt3,col5txt3
col1txt4,col2txt4,INFO,col4txt4,col5txt4
col1txt5,col2txt5,WARNING,col4txt5,col5txt5
col1txt6,col2txt6,ERROR,col4txt6,col5txt6
col1txt7,col2txt7,,col4txt7,col5txt7
col1txt8,col2txt8,INFO,col4txt8,col5txt8
col1txt9,col2txt9,INFO,col4txt9,col5txt9
col1txt10,col2txt10,ERROR,col4txt10,col5txt10
col1txt11,col2txt11,WARNING,col4txt11,col5txt11
col1txt12,col2txt12,WARNING,col4txt12,col5txt12
col1txt13,col2txt13,WARNING,col4txt13,col5txt13
col1txt14,col2txt14,WARNING,col4txt14,col5txt14
col1txt15,col2txt15,,col4txt15,col5txt15
col1txt16,col2txt16,,col4txt16,col5txt16
col1txt17,col2txt17,,col4txt17,col5txt17
col1txt18,col2txt18,,col4txt18,col5txt18
col1txt19,col2txt19,,col4txt19,col5txt19
col1txt20,col2txt20,,col4txt20,col5txt20
col1txt21,col2txt21,,col4txt21,col5txt21
col1txt22,col2txt22,,col4txt22,col5txt22
col1txt23,col2txt23,,col4txt23,col5txt23
col1txt24,col2txt24,,col4txt24,col5txt24
col1txt25,col2txt25,,col4txt25,col5txt25
col1txt26,col2txt26,,col4txt26,col5txt26
col1txt27,col2txt27,,col4txt27,col5txt27

预期输出文件 #

Header 1,Header 2,Severity,Header 4,Header 5
col1txt7,col2txt7,,col4txt7,col5txt7
col1txt15,col2txt15,,col4txt15,col5txt15
col1txt16,col2txt16,,col4txt16,col5txt16
col1txt17,col2txt17,,col4txt17,col5txt17
col1txt18,col2txt18,,col4txt18,col5txt18
col1txt19,col2txt19,,col4txt19,col5txt19
col1txt20,col2txt20,,col4txt20,col5txt20
col1txt21,col2txt21,,col4txt21,col5txt21
col1txt22,col2txt22,,col4txt22,col5txt22
col1txt23,col2txt23,,col4txt23,col5txt23
col1txt24,col2txt24,,col4txt24,col5txt24
col1txt25,col2txt25,,col4txt25,col5txt25
col1txt26,col2txt26,,col4txt26,col5txt26
col1txt27,col2txt27,,col4txt27,col5txt27

修正后的代码及解释 #

原代码存在3个核心问题：

1. 未指定CSV分隔符，需通过-F','明确逗号为分隔符
2. 变量传递格式错误，awk变量无需额外包裹${}，且定义时多了冗余逗号
3. awk中引用列需用$+列号，判断空值应使用!length($pos_sev)

基础版本（仅写入文件） #

awk -F',' -v pos_sev="$severity_column" 'NR==1 || !length($pos_sev)' "$source_log_file" > "$no_severity_logs_file"

带终端输出的版本（同时输出到终端和文件） #

awk -F',' -v pos_sev="$severity_column" 'NR==1 || !length($pos_sev)' "$source_log_file" | tee "$no_severity_logs_file"

关键细节说明 #

• -F','：指定CSV的字段分隔符为逗号
• -v pos_sev="$severity_column"：将shell变量severity_column的值传递给awk内部变量pos_sev
• NR==1 || !length($pos_sev)：保留表头（NR==1表示第一行），同时保留Severity列为空的行
• 使用>而非tee -a：避免多次运行时重复追加内容到目标文件

内容的提问来源于stack exchange，提问作者MarkoftheKanes