Nginx 1.2.5日志格式过滤请求敏感信息及自定义变量使用问题

Hey, let's break down your Nginx log issues step by step—you're hitting classic pitfalls with variable scoping, regex replacement, and how Nginx processes variables across request stages. Let's fix this properly.

一、为什么过滤敏感信息后日志只写空行？ #

Chances are your regex replacement was too aggressive, wiping out the entire $request value instead of just the sensitive parts. $request includes the HTTP method, URI, and protocol, so we need to target only the sensitive parameter values in the URI query string, not the whole line.

正确的敏感信息过滤配置 #

First, define a custom variable to hold the sanitized request (do this in your server or target location block, before your log_format):

# Create a copy of $request to modify
set $sanitized_request $request;

# Replace password values with ***
rewrite "(password=)[^&]+" $1*** break;
# Add more rules for other sensitive params like secret/api_key
rewrite "(secret=)[^&]+" $1*** break;
rewrite "(api_key=)[^&]+" $1*** break;

• The regex (password=)[^&]+ matches the parameter name (password=) plus all characters until the next & (end of the parameter value). We keep the parameter name and replace the value with ***.
• Using rewrite ... break ensures we modify the variable in the rewrite stage, which runs before the log stage.

If you prefer if statements over rewrite (they work the same here), use this instead:

set $sanitized_request $request;
if ($sanitized_request ~* "(password=)[^&]+") {
    set $sanitized_request $1***;
}
if ($sanitized_request ~* "(secret=)[^&]+") {
    set $sanitized_request $1***;
}

Then reference this sanitized variable in your log_format:

log_format main '$remote_addr - $remote_user [$time_local] "$sanitized_request" '
                '$status $body_bytes_sent "$http_referer" '
                '"$http_user_agent" "$http_x_forwarded_for"';

二、为什么自定义变量不显示，且$arg_password变空？ #

The root causes: #

1. Variable scoping/order: Custom variables must be defined before your log_format—Nginx processes configuration top-to-bottom, so if you define the variable after the log format, the log stage can't see it.
2. Accidentally overwriting system variables: $arg_password is an auto-generated variable Nginx creates from the request's query string. If you wrote something like set $arg_password "";, you overwrote the original value with an empty string.

正确的自定义变量配置 #

Let's fix your $yyy example and preserve $arg_password:

server {
    listen 80;
    server_name your-domain.com;

    # Define your custom variable FIRST, before log_format
    set $yyy 'abc';

    # Sanitize request (as above)
    set $sanitized_request $request;
    rewrite "(password=)[^&]+" $1*** break;

    # Define log_format with your custom variable
    log_format custom_log '$remote_addr - $yyy [$time_local] "$sanitized_request" $status';

    # Assign the log format to your access log
    access_log /var/log/nginx/your-domain.access.log custom_log;

    # Your location blocks go here
    location / {
        root /usr/share/nginx/html;
        index index.html;
    }
}

• By defining $yyy in the server block (before log_format), it's available to all requests in this server context.
• We're not touching $arg_password directly—we're only modifying the $sanitized_request copy, so the original $arg_password remains intact if you need it elsewhere.

验证变量是否生效 #

To confirm your variables work before checking logs, use the echo module (if you have it installed) to test:

location /test-vars {
    echo "Custom variable yyy: $yyy";
    echo "Sanitized request: $sanitized_request";
    echo "Original password arg: $arg_password";
}

Visit http://your-domain.com/test-vars?password=mysecret—you should see:

Custom variable yyy: abc
Sanitized request: GET /test-vars?password=*** HTTP/1.1
Original password arg: mysecret

If this works, your logs will too.

三、关键坑点要记住 #

• Test your regex: Always verify your regex doesn't match more than intended—use a regex tester to make sure you're only replacing sensitive values, not the entire request line.
• Variable order matters: Custom variables must be defined before the log_format that uses them.
• Don't overwrite auto-generated variables: Avoid set $arg_* or set $http_* unless you explicitly need to modify those values.
• Scope correctly: Variables defined in a location block only apply to that location; use the server block for global variables.

内容的提问来源于stack exchange，提问作者unional