OpenResty与Lua性能调优：定位高CPU耗时代码方案示例

Hey there! Let’s dig into troubleshooting your OpenResty 1.15.8.1 (Lua 5.2.4) performance issues—slow request handling and high CPU during validation are classic bottlenecks we can pinpoint with targeted tools. Below are practical, accurate methods to analyze function-level CPU usage, memory allocation, and latency, along with actionable examples:

1. Precision Timing for Function Latency #

Use OpenResty’s built-in ngx.now() to wrap your validation and request-handling functions, tracking exact execution time, call counts, and aggregated stats. This is great for pinpointing slow individual functions.

-- Initialize a global table to track performance stats
local perf_stats = {}

local function timeit(func, ...)
    local func_name = debug.getinfo(func).name or "anonymous"
    local start = ngx.now()
    -- Execute the function and capture its return values
    local result = {func(...)}
    local elapsed = (ngx.now() - start) * 1000 -- Convert to milliseconds

    -- Update stats for the function
    if not perf_stats[func_name] then
        perf_stats[func_name] = {count = 0, total_ms = 0, max_ms = 0}
    end
    local stats = perf_stats[func_name]
    stats.count = stats.count + 1
    stats.total_ms = stats.total_ms + elapsed
    stats.max_ms = math.max(stats.max_ms, elapsed)

    -- Print a summary every 100 requests to avoid log spam
    if stats.count % 100 == 0 then
        local avg_ms = stats.total_ms / stats.count
        ngx.log(ngx.INFO, string.format(
            "[PERF SUMMARY] %s: Calls=%d | Avg=%.3fms | Max=%.3fms",
            func_name, stats.count, avg_ms, stats.max_ms
        ))
    end

    return unpack(result)
end

-- Wrap your validation function with timeit when calling it
local request_data = ngx.req.get_body_data()
local is_valid = timeit(your_validation_function, request_data)

2. CPU Sampling to Identify Hot Functions #

Lua’s debug.sethook lets you sample the call stack periodically, revealing which functions consume the most CPU. This is ideal for finding CPU-intensive code segments without adding heavy overhead.

local sample_interval = 0.01 -- Sample every 10ms (adjust based on needs)
local sample_counts = {}

-- Hook function to count active functions during sampling
local function sample_hook()
    local call_info = debug.getinfo(2, "n") -- Get the calling function
    if call_info and call_info.name then
        sample_counts[call_info.name] = (sample_counts[call_info.name] or 0) + 1
    end
end

-- Start sampling (run this in init_worker_by_lua* or your request handler)
debug.sethook(sample_hook, "c", sample_interval * 1000000) -- Convert to microseconds

-- Schedule a periodic report to log CPU distribution
local function report_cpu_samples()
    ngx.log(ngx.INFO, "[CPU SAMPLE REPORT]")
    local total_samples = 0
    for _, count in pairs(sample_counts) do
        total_samples = total_samples + count
    end
    -- Sort functions by sample count (descending)
    local sorted_funcs = {}
    for name, count in pairs(sample_counts) do
        table.insert(sorted_funcs, {name = name, count = count})
    end
    table.sort(sorted_funcs, function(a, b) return a.count > b.count end)
    -- Print results
    for _, func in ipairs(sorted_funcs) do
        local percentage = (func.count / total_samples) * 100
        ngx.log(ngx.INFO, string.format("  %s: %.2f%% (%d samples)", func.name, percentage, func.count))
    end
    -- Reset counts for next interval
    sample_counts = {}
    return true
end

-- Run the report every 60 seconds
ngx.timer.every(60, report_cpu_samples)

Note: Sampling adds minor overhead—disable it once you’ve identified bottlenecks, or increase the interval in production.

3. Memory Allocation Tracking #

Use Lua’s garbage collector functions to measure how much memory a function allocates, helping spot leaks or excessive memory usage during validation.

local function track_memory(func, ...)
    -- Force garbage collection to get a clean baseline
    collectgarbage("collect")
    local start_mem_kb = collectgarbage("count") -- Returns KB used
    -- Execute the function
    local result = {func(...)}
    -- Collect garbage again to measure net allocation
    collectgarbage("collect")
    local end_mem_kb = collectgarbage("count")
    local mem_diff_kb = end_mem_kb - start_mem_kb

    ngx.log(ngx.INFO, string.format(
        "[MEM TRACK] %s allocated %.2f KB",
        debug.getinfo(func).name or "anonymous", mem_diff_kb
    ))

    return unpack(result)
end

-- Use it with your validation function
local is_valid = track_memory(your_validation_function, request_data)

4. System-Level Profiling with perf #

For deeper insights (including interactions between Lua code and Nginx’s C core), use the Linux perf tool to sample your OpenResty worker process:

# Find the PID of your OpenResty worker process
ps aux | grep nginx | grep worker

# Sample CPU usage for 30 seconds, capturing call graphs
perf record -p <WORKER_PID> -g -F 99 -- sleep 30

# Generate a human-readable report
perf report

This will show you which Lua functions (and Nginx internal functions) are consuming the most CPU, even for low-level operations.

5. Offline Testing with resty-cli #

If your validation logic can run independently, use resty-cli’s profiling mode to get a quick breakdown of function performance in a development environment:

# Run your validation script with profiling enabled
resty --profile your_validation_logic.lua

The output will include call counts, total time, and average time per function call—perfect for iterative testing.

Key Tips for Production #

• Avoid long-running debug tools: All profiling methods add overhead—enable them only during troubleshooting windows.
• Prioritize validation logic: Focus your analysis on functions tied to request validation (signature checks, parameter parsing, external auth calls) since that’s where you’ve observed issues.
• Check for blocking IO: If latency comes from waiting on external services (e.g., Redis, databases), switch to async patterns like resty.http’s async requests or ngx.timer to avoid blocking the worker process.

内容的提问来源于stack exchange，提问作者Abdol Seed