如何从result.txt中移除ips.yaml包含的IP对应行

我有两个文件：

• ips.yaml 内容如下：

-
  dedicatedip: 1.1.1.11
-
  dedicatedip: 2.2.2.2
-
  dedicatedip: ''
-
  dedicatedip: 3.3.3.3
-
  dedicatedip: 3.3.3.33

• result.txt 是表格形式，内容如下：

+----------+-----------------------+------------------+--------+-----------------------------+---------------+
| id       | hostname              | some other field | status | networks                    | plan          |
+----------+-----------------------+------------------+--------+-----------------------------+---------------+
| 11111111 | some-hostname         | some-value       | Active | External_Networks=1.1.1.11  | not important |
| 11111111 | some another hostname |                  | Active | External_Networks=1.1.1.111 | not important |
| 11111111 | some.fqdn.com         |                  | Active | External_Networks=1.1.1.112 | not important |
| 11111111 | fourth.hostname.com   |                  | Active | External_Networks=1.1.1.1   | not important |
| 11111111 | the-other.com         | IHaveSomething   | Active | External_Networks=2.2.2.2   | not important |
| 11111111 | not.least.com         |                  | Active | External_Networks=2.2.2.25  | not important |
| 11111111 | last.fqdn.com         | The Last Value   | Active | External_Networks=3.3.3.39  | not important |
+----------+-----------------------+------------------+--------+-----------------------------+---------------+

需求是移除result.txt中包含ips.yaml内IP的行，预期输出：

+----------+-----------------------+------------------+--------+-----------------------------+---------------+
| id       | hostname              | some other field | status | networks                    | plan          |
+----------+-----------------------+------------------+--------+-----------------------------+---------------+
| 11111111 | some another hostname |                  | Active | External_Networks=1.1.1.111 | not important |
| 11111111 | some.fqdn.com         |                  | Active | External_Networks=1.1.1.112 | not important |
| 11111111 | fourth.hostname.com   |                  | Active | External_Networks=1.1.1.1   | not important |
| 11111111 | not.least.com         |                  | Active | External_Networks=2.2.2.25  | not important |
| 11111111 | last.fqdn.com         | The Last Value   | Active | External_Networks=3.3.3.39  | not important |
+----------+-----------------------+------------------+--------+-----------------------------+---------------+

我当前的bash脚本：

while read -r yaml_ips
do
    while read -r result_ips
    do
        if [ "$yaml_ips" == "$result_ips" ]
        then
            sed "/$yaml_ips/d" result.txt
        fi
    done < <(grep -Eo 'External_Networks=[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' result.txt | awk -F '=' '{print $2}')
done < <(awk '/dedicatedip: [0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}/ {print $2}' ips.yaml)

但输出不符合预期，内容重复且没正确移除目标行：

+----------+-----------------------+------------------+--------+-----------------------------+---------------+
| id       | hostname              | some other field | status | networks                    | plan          |
+----------+-----------------------+------------------+--------+-----------------------------+---------------+
+----------+-----------------------+------------------+--------+-----------------------------+---------------+
+----------+-----------------------+------------------+--------+-----------------------------+---------------+
| id       | hostname              | some other field | status | networks                    | plan          |
+----------+-----------------------+------------------+--------+-----------------------------+---------------+
| 11111111 | some-hostname         | some-value       | Active | External_Networks=1.1.1.11  | not important |
| 11111111 | some another hostname |                  | Active | External_Networks=1.1.1.111 | not important |
| 11111111 | some.fqdn.com         |                  | Active | External_Networks=1.1.1.112 | not important |
| 11111111 | fourth.hostname.com   |                  | Active | External_Networks=1.1.1.1   | not important |
| 11111111 | last.fqdn.com         | The Last Value   | Active | External_Networks=3.3.3.39  | not important |
+----------+-----------------------+------------------+--------+-----------------------------+---------------+

原脚本存在两个核心问题：

1. 重复输出文件内容：每次匹配到IP就执行sed "/$yaml_ips/d" result.txt，会把整个文件处理一遍并输出，导致多次重复打印文件内容。
2. 逻辑冗余效率低：嵌套循环逐行比对IP完全没必要，sed或awk可以一次性批量处理所有需要排除的IP。

方法1：使用awk一次性完成（推荐） #

awk -F'=' '
    NR == FNR {
        if ($2 ~ /^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$/) {
            exclude[$2] = 1
        }
        next
    }
    {
        split($5, net, "=")
        if (!(net[2] in exclude)) {
            print $0
        }
    }
' ips.yaml result.txt

方法2：使用grep结合临时文件 #

# 提取ips.yaml中的有效IP，生成排除模式
grep -E 'dedicatedip: [0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' ips.yaml | awk '{print $2}' > exclude_ips.txt

# 排除包含这些IP的行，保留表头和分隔线
grep -v -f <(sed 's/^/External_Networks=/' exclude_ips.txt) result.txt

# 清理临时文件
rm exclude_ips.txt

• 方法1：用awk同时读取两个文件，先把ips.yaml中的有效IP存入排除数组，再遍历result.txt，检查每行的IP是否在排除数组中，不在则打印。
• 方法2：先提取需要排除的IP，转换成External_Networks=IP的格式，再用grep -v排除匹配的行，简单直观。

内容的提问来源于stack exchange，提问作者Saeed