使用awk实现示例文件单词计数并输出符合指定规则的统计结果

awk脚本应当生成统计表格，列出长度超过4个字母且出现次数大于2次的单词，输出的最后一行需展示文件的总单词数。

输入文件内容 #

In navigation, the heading of a vessel or aircraft is the compass direction
in which the craft's bow or nose is pointed. Note that the heading may not
#necessarily be the direction that the vehicle actually travels, which is
known as its course or track. Any difference between the heading and
course is due to the motion of the underlying medium, the air or water,
or other effects like skidding or slipping. The difference is known as
the drift, and can be determined by the wind triangle. At least seven ways
to measure the heading of a vehicle have been described. A compass installed
in a vehicle or vessel has a certain amount of error caused by the magnetic 
properties of the vessel. This error is known as compass deviation. The 
magnitude of the compass deviation varies greatly depending upon the local 
anomalies created by the vessel. A fiberglass recreational vessel will 
#generally have much less compass deviation than a steel-hulled vessel. 
Electrical wires carrying current have a small magnetic field around them 
and can cause deviation. Any type of magnet, such as found in a speaker 
can also cause large magnitudes of compass deviation. The error can be 
corrected using a deviation table. Deviation tables are very difficult to 
create. Once a deviation table is established, it is only good for that 
particular vessel, with that particular configuration. If electrical wires 
are moved or anything else magnetic (speakers, electric motors, etc.) are 
moved, the deviation table will change. All deviations in the deviation 
table are indicated west or east. If the compass is pointing west of the 
Magnetic North Pole, then the deviation is westward. If the compass is 
pointing east of the Magnetic North Pole, then the deviation is eastward.

任务要求 #

• 编写awk脚本从输入文件生成统计报告，统计输入文件中每个单词的出现次数。
• 额外实现忽略.、,:()".*等标点符号的功能。

参考实现（存在缺陷） #

现有参考命令仅实现了基础的单词计数，未处理标点符号、过滤统计条件和总单词数统计逻辑：

cat input.txt|awk -F" " '{for(i=1;i<=NF;i++) a[$i]++} END {for(k in a) print k,a[k]}'

最终可用awk脚本 #

awk '{
    # 遍历每一行的所有字段
    for(i=1;i<=NF;i++) {
        # 移除所有需要忽略的标点符号
        word = $i
        gsub(/[.、,:()#*"]/, "", word)
        # 跳过处理后为空的内容
        if(word == "") continue
        # 统计单词出现次数
        count[word]++
        # 累加总单词数
        total++
    }
}
END {
    # 输出符合条件的单词：长度>4，出现次数>2
    for(word in count) {
        if(length(word) >4 && count[word] >2) {
            printf "%-20s %d\n", word, count[word]
        }
    }
    # 最后一行输出总单词数
    printf "总单词数: %d\n", total
}' input.txt

脚本逻辑说明 #

1. 用gsub函数批量移除所有指定的标点符号，避免标点被计入单词内容
2. 单次扫描文件同时完成单词计数和总单词数累加，执行效率更高
3. END块先筛选符合长度和出现次数要求的单词输出，最后打印总单词数

内容的提问来源于stack exchange，提问作者Sara Jeon