Bash下正则匹配IP调用API替换Apache日志端口为国家名的实现方法

原始资料 #

Apache错误日志样例 #

[Wed Sep 01 15:37:10.709437 2021] [authz_core:error] [pid 15732] [client [IP REDACTED]:56163] AH01630: client denied by server configuration: /var/www/[REDACTED].com/html/wp-admin/admin-post.php

IP地理查询命令 #

curl 'https://api.ipgeolocation.io/ipgeo?apiKey=[REDACTED]&ip=[REDACTED]'

API返回JSON样例 #

{"ip":"[REDACTED]","continent_code":"EU","continent_name":"Europe","country_code2":"DE","country_code3":"DEU","country_name":"Germany","country_capital":"Berlin","state_prov":"Bayern","district":"Oberbayern","city":"München","zipcode":"81549","latitude":"[REDACTED]","longitude":"[REDACTED]","is_eu":true,"calling_code":"+49","country_tld":".de","languages":"de","country_flag":"https://ipgeolocation.io/static/flags/de_64.png","geoname_id":"8772423","isp":"[REDACTED]","connection_type":"","organization":"[REDACTED]","currency":{"code":"EUR","name":"Euro","symbol":"€"},"time_zone":{"name":"Europe/Berlin","offset":1,"current_time":"2021-09-02 23:08:55.514+0200","current_time_unix":1630616935.514,"is_dst":true,"dst_savings":1}}

现有正则匹配命令 #

grep -P "\b(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?):[0-9]+\b" ~/Documents/hosting/web/logscript_results.txt

预期输出效果 #

[Wed Sep 01 15:37:10.709437 2021] [authz_core:error] [pid 15732] [client [IP REDACTED]: Germany AH01630: client denied by server configuration: /var/www/[REDACTED].com/html/wp-admin/admin-post.php

具体实现 #

grep仅能做匹配无法直接完成替换和API联动，你可以用bash脚本结合jq完成需求，完全兼容你现有正则规则：

1. 先安装JSON解析工具jq：
   • Debian/Ubuntu系统：sudo apt install jq
   • macOS系统：brew install jq
2. 运行如下脚本即可生成处理后的日志：

#!/bin/bash
# 替换为你自己的apiKey
API_KEY="你的实际API密钥"
LOG_PATH="$HOME/Documents/hosting/web/logscript_results.txt"
# 关联数组做IP查询缓存，避免重复请求API
declare -A IP_CACHE

while read -r line; do
    # 完全复用你现有正则，新增分组捕获IP和完整匹配项
    if [[ $line =~ \b((?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)):([0-9]+)\b ]]; then
        ip="${BASH_REMATCH[1]}"
        full_match="${BASH_REMATCH[0]}"
        # 缓存未命中时调用API查询
        if [[ ! -v IP_CACHE[$ip] ]]; then
            country=$(curl -s "https://api.ipgeolocation.io/ipgeo?apiKey=${API_KEY}&ip=${ip}" | jq -r '.country_name')
            IP_CACHE[$ip]="$country"
        fi
        # 替换端口为国家名输出
        echo "${line//$full_match/$ip: ${IP_CACHE[$ip]}}"
    else
        # 无匹配的行原样输出
        echo "$line"
    fi
done < "$LOG_PATH"

内容的提问来源于stack exchange，提问作者lampstackxyz