如何用awk循环对比两文件IP关联的到期日期并调用API

现有三个文件：file-old.txt、file-new.txt、servers.txt，内容如下：

file-old.txt #

nextduedate: '2023-09-06'
dedicatedip: 1.1.1.1
nextduedate: '2023-09-01'
dedicatedip: 2.2.2.2
nextduedate: '2023-09-08'
dedicatedip: 3.3.3.3

file-new.txt #

nextduedate: '2023-09-06'
dedicatedip: 1.1.1.1
nextduedate: '2023-10-01'
dedicatedip: 2.2.2.2
nextduedate: '2023-10-08'
dedicatedip: 3.3.3.3

servers.txt #

[
    {
        "server": {
            "server_ip": "3.3.3.3",
            "server_ipv6_net": "::1", # fake
            "server_number": 12345678974,
            "server_name": "some.host.name.com",
            "product": "Server Auction",
            "dc": "FSN1-DC11",
            "traffic": "unlimited",
            "status": "ready",
            "cancelled": false,
            "paid_until": "2023-10-08",
            "ip": [
                "3.3.3.3"
            ],
            "subnet": [
                {
                    "ip": "::1", # fake
                    "mask": "64" # fake
                }
            ],
            "linked_storagebox": null
        }
    },
# other server's details
]

需求 #

• 对比两个文件中同一IP对应的nextduedate
• 若file-new.txt中的日期晚于file-old.txt，从servers.txt获取该IP对应的server_number，执行API调用：curl -u user:password https://url.com/$server_number
• 日期相同则不执行任何操作

当前问题 #

原有awk写法会让旧文件的每个IP与新文件的所有IP逐一对比，导致重复匹配（比如1.1.1.1会和新文件的所有IP对比一遍），逻辑冗余且效率低下。

核心是用关联数组（键值对）存储每个IP对应的日期，直接通过IP匹配对应日期，避免无意义的全量对比。

步骤1：提取IP与server_number的映射 #

由于servers.txt是JSON格式，用jq提取更可靠（如果没有jq，也可以用awk）：

# 用jq提取（推荐，处理标准JSON更稳定）
jq -r '.[] | .server.server_ip + " " + (.server.server_number|tostring)' servers.txt > ip-server-map.txt

# 若没有jq，用awk提取
awk -F'"' '/"server_ip":/{ip=$4} /"server_number":/{num=$4; print ip, num}' servers.txt > ip-server-map.txt

执行后会生成ip-server-map.txt，每行格式为IP server_number。

步骤2：主脚本对比日期并执行API调用 #

awk '
# 处理旧文件，构建IP到旧日期的映射
FILENAME == ARGV[1] {
    if ($1 == "nextduedate:") {
        gsub(/'\''/,"",$2); old_date = $2
    }
    if ($1 == "dedicatedip:") {
        ip = $2; old_dates[ip] = old_date
    }
    next
}
# 处理新文件，构建IP到新日期的映射
FILENAME == ARGV[2] {
    if ($1 == "nextduedate:") {
        gsub(/'\''/,"",$2); new_date = $2
    }
    if ($1 == "dedicatedip:") {
        ip = $2; new_dates[ip] = new_date
    }
    next
}
# 处理IP-server映射文件，构建IP到server_number的映射
FILENAME == ARGV[3] {
    ip_server_map[$1] = $2
    next
}
# 遍历所有IP，对比日期并执行操作
END {
    for (ip in old_dates) {
        # 只处理新旧文件都存在的IP
        if (ip in new_dates) {
            # YYYY-MM-DD格式的日期可直接用字符串比较大小
            if (new_dates[ip] > old_dates[ip]) {
                server_num = ip_server_map[ip]
                if (server_num != "") {
                    cmd = "curl -u user:password https://url.com/" server_num
                    print "执行命令: " cmd
                    system(cmd)
                } else {
                    print "警告: IP " ip  " 未找到对应的server_number"
                }
            }
        }
    }
}
' file-old.txt file-new.txt ip-server-map.txt

关键说明 #

• 用关联数组old_dates、new_dates存储IP与对应日期的映射，直接通过IP键匹配，避免全量对比
• YYYY-MM-DD格式的日期字符串可直接用>比较大小，无需额外转换为时间戳
• 先提取IP-server映射，避免主脚本重复解析复杂的JSON结构，提升效率

内容的提问来源于stack exchange，提问作者Saeed