如何优化大文件grep匹配，仅检索脚本执行后的新内容？

Hey there! Let's tackle your WebLogic server restart script issues step by step. The core problems here are matching old log entries in a huge, unrotated log file and inefficiently scanning the entire file—we'll fix both with targeted tweaks and better approaches.

1. Stop Scanning Old Logs: Track Only New Content #

Your current fgrep checks the entire log file, which is why it's picking up old "accepting" entries from previous restarts. Instead, we need to only monitor new log lines generated after your script starts the server.

Use tail -n 0 -f to start reading from the end of the log file (ignoring all existing content) and follow new entries as they're added. Pair this with grep -m 1 to stop immediately after the first match of your target string—this is way more efficient for large files.

Here's the revised loop for starting N2:

#startN2
nohup sh startstopAIA2.sh start > N2Start.out &
sleep 1s

# Wait for the first new "accepting" entry in the log
echo "Waiting for N2 to finish starting..."
tail -n 0 -f /path/logs/AIAPPD2D_SOA2.out | grep -m 1 "SOA Platform is running and accepting requests"

sleep 0.5s
echo "N2 is running. Proceeding to restart N1"
#stopN1

• tail -n 0: Starts reading from the current end of the log (no old content)
• -f: Follows new lines as they're written
• grep -m 1: Stops grep as soon as the first match is found, which saves CPU cycles on huge files

2. Bonus: Make the Log Monitoring More Robust #

If you're worried about tail hanging indefinitely if the server fails to start, you can add a timeout with the timeout command (available on most Linux systems):

# Wait up to 10 minutes (600 seconds) for the server to start
timeout 600 tail -n 0 -f /path/logs/AIAPPD2D_SOA2.out | grep -m 1 "SOA Platform is running and accepting requests"

# Check if the grep succeeded (exit code 0 means match found)
if [ $? -ne 0 ]; then
    echo "ERROR: N2 failed to start within 10 minutes. Exiting script."
    exit 1
fi

This prevents your script from hanging forever if something goes wrong with the server startup.

3. Long-Term Fix: Fix Log Rotation #

The root cause of your large log file is that it's not being rotated. Configure logrotate for your WebLogic logs to keep them manageable:

• Create a logrotate config file (e.g., /etc/logrotate.d/weblogic_soa):

/path/logs/AIAPPD2D_SOA2.out {
    daily
    rotate 7
    compress
    missingok
    notifempty
    copytruncate
}

This will rotate the log daily, keep 7 days of history, compress old logs, and truncate the original log after copying (so WebLogic doesn't need to restart to write to a new file).

4. Even Better: Use WebLogic's API for Status Checks #

Log parsing can be fragile (e.g., if the log message changes slightly). For a more reliable approach, use WebLogic's WLST (WebLogic Scripting Tool) to check if the managed server is in RUNNING state:

# Sample WLST script to check server status
cat << EOF > check_server_status.py
connect('admin_user', 'admin_pass', 't3://admin-server:7001')
serverStatus('Node2')
exit()
EOF

# Wait until server is running
until java weblogic.WLST check_server_status.py | grep -m 1 "RUNNING"; do
    sleep 5
done

This is a more robust method because it queries the WebLogic Admin Server directly, avoiding any log parsing issues. Just replace the admin credentials and server URL with your actual values.

内容的提问来源于stack exchange，提问作者pratik swami