如何在Julia中独立于Optim.jl使用Linesearches.jl模块？

Got it, I totally get the frustration when package docs only cover integrated use cases instead of standalone usage. Let’s break down how to use Linesearches.jl with your custom L-BFGS implementation step by step.

Step 1: Setup Dependencies and Core Functions #

First, make sure you've installed the required packages, then define your objective function and gradient (we'll use the Rosenbrock function as a common test case):

using LinearAlgebra, Linesearches

# Example objective function: Rosenbrock
function f(x)
    return (1.0 - x[1])^2 + 100.0 * (x[2] - x[1]^2)^2
end

# In-place gradient function (more efficient for large problems)
function ∇f!(g, x)
    g[1] = -2.0 * (1.0 - x[1]) - 400.0 * (x[2] - x[1]^2) * x[1]
    g[2] = 200.0 * (x[2] - x[1]^2)
    return g
end

Step 2: Line Search Basics for Your L-BFGS Loop #

Linesearches.jl's core functions (like backtracking! or strong_wolfe!) work by taking your current iterate, search direction, gradient, and function value, then finding the optimal step size. You don't need to tie it to Optim.jl at all—just pass the right inputs.

Example 1: Backtracking Line Search #

This is a robust, simple option for most cases:

# Initialize variables for your L-BFGS iteration
x = [0.0, 0.0]
g = similar(x)
∇f!(g, x)
fx = f(x)

# Simulate getting a search direction from your L-BFGS implementation
# (replace this with your actual L-BFGS direction calculation)
d = -g

# Configure backtracking parameters
ls_config = Backtracking(alpha0=1.0, rho=0.5, c1=1e-4)

# Run the line search
alpha, fx_new, x_new, g_new = backtracking!(f, ∇f!, x, d, g, fx, ls_config)

Example 2: Strong Wolfe Line Search #

For faster convergence in smooth problems, use the strong Wolfe conditions:

# Configure strong Wolfe parameters
ls_config = StrongWolfe(alpha0=1.0, c1=1e-4, c2=0.9, maxiter=100)

# Run the line search (note the extra return values for step-specific gradients/function values)
alpha, fx_new, x_new, g_new, f_alpha, g_alpha = strong_wolfe!(f, ∇f!, x, d, g, fx, ls_config)

Step 3: Integrate into Your Full L-BFGS Implementation #

Here's a simplified framework showing how to slot the line search into your custom L-BFGS loop:

function custom_lbfgs(f, ∇f!, x0; maxiter=100, line_search=Backtracking())
    x = copy(x0)
    g = similar(x)
    ∇f!(g, x)
    fx = f(x)
    
    # Initialize L-BFGS history (replace with your actual state storage for s/y pairs)
    lbfgs_history = []

    for iter in 1:maxiter
        # 1. Compute L-BFGS search direction (replace with your logic)
        d = -g  # Placeholder: replace with actual L-BFGS direction calculation
        
        # 2. Run line search to find optimal step size
        alpha, fx_new, x_new, g_new = backtracking!(f, ∇f!, x, d, g, fx, line_search)
        
        # 3. Update L-BFGS history (curvature pairs s = x_new - x, y = g_new - g)
        s = x_new - x
        y = g_new - g
        push!(lbfgs_history, (s, y))
        # Trim history if you're limiting the number of stored pairs (standard L-BFGS practice)
        
        # 4. Update current iteration state
        x, fx, g = x_new, fx_new, g_new
        
        # 5. Convergence check
        if norm(g) < 1e-6
            println("Converged at iteration $iter")
            break
        end
    end
    return x, fx
end

# Test the custom L-BFGS with different line searches
x_opt_backtrack, fx_opt_backtrack = custom_lbfgs(f, ∇f!, [0.0, 0.0])
x_opt_wolfe, fx_opt_wolfe = custom_lbfgs(f, ∇f!, [0.0, 0.0], line_search=StrongWolfe())

Key Notes for Tuning #

• Line Search Parameters: Adjust alpha0 (initial step guess), c1/c2 (Wolfe condition constants), and rho (backtracking shrink factor) based on your problem's characteristics.
• Return Values: Different line search functions have slightly different return signatures—check the Linesearches.jl source code (src/linesearches.jl) for exact details if you need more outputs.
• History Management: Make sure your L-BFGS implementation correctly updates its curvature pair history after each line search step, as this is critical for computing accurate search directions.

内容的提问来源于stack exchange，提问作者Simon Batzner