Bash别名定义的awk去重命令与直接执行结果不一致问题排查

问题描述 #

在Ubuntu 20.04中，我使用以下awk命令对gigs.csv按第二列去重，执行结果符合预期：

awk -F, ' {key = $2 FS} !seen[key]++' gigs.csv > try.csv

为避免重复输入，我在~/.bash_aliases中定义了别名：

alias dedupe="awk -F, ' {key = $2 FS} !seen[key]++' gigs.csv > try.csv"

但运行dedupe时仅输出一行，与直接执行原命令的结果不符。

gigs.csv示例内容：

Tue 30 Aug 08:34:17 AM,Do you use facebook? work remote from home. we are hiring!,https://atlanta.craigslist.org/atl/cpg/d/atlanta-do-you-use-facebook-work-remote/7527729597.html
Mon 29 Aug 03:51:29 PM,Cash for your opinions!,https://atlanta.craigslist.org/atl/cpg/d/atlanta-cash-for-your-opinions/7527517063.html
Mon 29 Aug 01:22:54 PM,Telecommute earn $20 per easy online product test gig w/ free products,https://montgomery.craigslist.org/cpg/d/hope-hull-telecommute-earn-20-per-easy/7527471859.html
Mon 29 Aug 01:53:58 PM,Telecommute earn $20 per easy online product test gig w/ free products,https://atlanta.craigslist.org/atl/cpg/d/smyrna-telecommute-earn-20-per-easy/7527456060.html
Mon 29 Aug 12:50:59 PM,Telecommute earn $20 per easy online product test gig w/ free products,https://bham.craigslist.org/cpg/d/adamsville-telecommute-earn-20-per-easy/7527454527.html
Wed 31 Aug 09:23:41 PM,Looking for a sales development rep,https://bham.craigslist.org/cpg/d/adamsville-looking-for-sales/7528472497.html
Wed 31 Aug 11:21:58 AM,Earn ~$30 | work from home | looking for 'ok google' users | taskverse,https://bham.craigslist.org/cpg/d/harbor-city-earn-30-work-from-home/7528233394.html
Mon 29 Aug 12:50:59 PM,Telecommute earn $20 per easy online product test gig w/ free products,https://bham.craigslist.org/cpg/d/adamsville-telecommute-earn-20-per-easy/7527454527.html
Wed 31 Aug 11:28:56 AM,Earn ~$30 | work from home | looking for 'ok google' users | taskverse,https://tuscaloosa.craigslist.org/cpg/d/harbor-city-earn-30-work-from-home/7528236901.html
Wed 31 Aug 11:27:53 AM,Earn ~$30 | work from home | looking for 'ok google' users | taskverse,https://montgomery.craigslist.org/cpg/d/harbor-city-earn-30-work-from-home/7528236389.html

问题原因 #

别名定义存在错误，核心是bash的引号解析规则：

• 外层使用双引号时，bash会优先解析内部的$2，将其视为当前shell的位置参数（通常为空值）。
• 这导致实际执行的awk命令中，$2被替换为空，变成awk -F, ' {key = FS} !seen[key]++' gigs.csv > try.csv。此时所有行的key值都是分隔符FS，自然只会保留第一行。

解决方法 #

有两种可行方案修复别名：

方案1：外层改用单引号 #

将别名的外层双引号替换为单引号，bash不会解析内部的$2，直接传递给awk。同时注意将awk脚本的单引号换成双引号，避免语法冲突：

alias dedupe='awk -F, " {key = $2 FS} !seen[key]++" gigs.csv > try.csv'

方案2：转义$2 #

保留外层双引号，但对$2添加反斜杠转义，告诉bash不要解析这个变量：

alias dedupe="awk -F, ' {key = \$2 FS} !seen[key]++' gigs.csv > try.csv"

生效验证 #

修改别名后，执行以下命令让别名立即生效：

source ~/.bash_aliases

再运行dedupe，就能得到和原命令一致的结果——保留每个第二列值的第一行，示例中最终try.csv会有6行内容。

内容的提问来源于stack exchange，提问作者Tendekai Muchenje