Shell脚本解析JSON无法提取失败状态siteIds，求解决方法

编写了一段Shell脚本，旨在遍历JSON响应的每一行，将所有状态为Failed的条目对应的siteIds提取为单个逗号分隔的字符串。但运行脚本后无法正确获取并拼接对应的siteIds，仅输出空的逗号。

原脚本 #

input='[
  {
    "siteIds": "93970,168794,34583,96844,34343,29095,98786,136728,33441,35913",
    "status": "Failed for some reason"
  },
  {
    "siteIds": "28709,29810,84178,31027,29985,100487,35486,36068,37001,36360",
    "status": "Failed for reason xyz"
  },
  {
    "siteIds": "190873,28714,93455,35995,12542,77957,102080,32006,109115,28902",
    "status": "Success"
  },
  {
    "siteIds": "30514,33349,29178,77989,30460,35541,36765,29637,97585,29263",
    "status": "Failed:..."
  },
  {
    "siteIds": "36913,36741,30435,35007,109238,30606,33669,76823,75882,30265",
    "status": "Success"
  }
]'

siteIds=""
while read -r line; do
  if [[ $line = *"status\": \"Failed"* ]]; then
    echo $line
    siteId=$(echo $line | grep -o '"siteIds": "[^"]*"' | awk -F': ' '{print $2}' | tr -d '"')
    siteIds="$siteIds$siteId,"
  fi
done <<< "$input"

if [[ -z "$siteIds" ]]; then
    siteIds = "No failed siteIds found."
fi

echo "${siteIds%?}"
echo "finished"

当前错误输出 #

"status": "Failed for some reason"
"status": "Failed for reason xyz"
"status": "Failed:..."
,,
finished

期望输出 #

93970,168794,34583,96844,34343,29095,98786,136728,33441,35913,28709,29810,84178,31027,29985,100487,35486,36068,37001,36360,30514,33349,29178,77989,30460,35541,36765,29637,97585,29263

原脚本的核心错误是：当匹配到status为Failed的行时，尝试从当前行提取siteIds，但siteIds和status属于同一个JSON对象的不同行，当前行根本没有siteIds的内容，因此grep无法匹配到结果，导致siteId为空，最终拼接出空字符串加逗号的结果。

方案1：使用jq处理JSON（推荐） #

处理JSON数据最可靠的方式是使用专门的JSON工具jq，避免行处理带来的格式兼容性问题。

修正后的脚本 #

input='[
  {
    "siteIds": "93970,168794,34583,96844,34343,29095,98786,136728,33441,35913",
    "status": "Failed for some reason"
  },
  {
    "siteIds": "28709,29810,84178,31027,29985,100487,35486,36068,37001,36360",
    "status": "Failed for reason xyz"
  },
  {
    "siteIds": "190873,28714,93455,35995,12542,77957,102080,32006,109115,28902",
    "status": "Success"
  },
  {
    "siteIds": "30514,33349,29178,77989,30460,35541,36765,29637,97585,29263",
    "status": "Failed:..."
  },
  {
    "siteIds": "36913,36741,30435,35007,109238,30606,33669,76823,75882,30265",
    "status": "Success"
  }
]'

# 使用jq提取符合条件的siteIds并拼接成逗号分隔字符串
result=$(echo "$input" | jq -r '.[] | select(.status | startswith("Failed")) | .siteIds' | tr '\n' ',' | sed 's/,$//')

# 处理无失败条目的情况
if [[ -z "$result" ]]; then
  echo "No failed siteIds found."
else
  echo "$result"
fi

命令解释 #

• jq -r '.[] | select(.status | startswith("Failed")) | .siteIds'：遍历JSON数组的每个元素，筛选status以Failed开头的条目，提取对应的siteIds并输出原始字符串（不带引号）。
• tr '\n' ','：将每个siteIds的换行符替换为逗号。
• sed 's/,$//'：去掉最后一个多余的逗号。

方案2：修改原Shell循环逻辑 #

如果无法使用jq，可以调整循环逻辑，先临时存储当前条目的siteIds，当匹配到Failed状态时再将其拼接到结果中。

修正后的脚本 #

input='[
  {
    "siteIds": "93970,168794,34583,96844,34343,29095,98786,136728,33441,35913",
    "status": "Failed for some reason"
  },
  {
    "siteIds": "28709,29810,84178,31027,29985,100487,35486,36068,37001,36360",
    "status": "Failed for reason xyz"
  },
  {
    "siteIds": "190873,28714,93455,35995,12542,77957,102080,32006,109115,28902",
    "status": "Success"
  },
  {
    "siteIds": "30514,33349,29178,77989,30460,35541,36765,29637,97585,29263",
    "status": "Failed:..."
  },
  {
    "siteIds": "36913,36741,30435,35007,109238,30606,33669,76823,75882,30265",
    "status": "Success"
  }
]'

siteIds=""
current_siteIds=""

while read -r line; do
  # 遇到siteIds行时，提取并临时存储
  if [[ "$line" == *"siteIds"* ]]; then
    current_siteIds=$(echo "$line" | grep -o '"siteIds": "[^"]*"' | awk -F': ' '{print $2}' | tr -d '"')
  fi
  # 遇到Failed状态行时，将临时存储的siteIds拼接到结果
  if [[ "$line" == *"status\": \"Failed"* ]]; then
    siteIds="$siteIds$current_siteIds,"
  fi
done <<< "$input"

# 处理结果
if [[ -z "$siteIds" ]]; then
  siteIds="No failed siteIds found."
else
  # 移除最后一个多余的逗号
  siteIds="${siteIds%?}"
fi

echo "$siteIds"
echo "finished"

修改点说明 #

• 新增current_siteIds变量，用于临时存储当前JSON对象的siteIds。
• 循环中先检查并提取siteIds行的内容，保存到临时变量。
• 当匹配到Failed状态行时，将临时变量中的siteIds拼接到最终结果字符串中。

内容的提问来源于stack exchange，提问作者Gojo