如何利用已知点函数值加速scipy.optimize.minimize带约束优化？

Great question—dealing with expensive objective functions is always a pain, especially when you already have some precomputed data sitting there that you know could help. Let’s break down the most practical ways to speed up scipy.optimize.minimize (especially with bound constraints) using your existing function values:

First and foremost, you want to avoid re-calculating the function for any point you’ve already evaluated—whether it’s your precomputed points or ones the optimizer generates along the way.

The easiest way to do this is with a cache. Since scipy.optimize.minimize passes numpy arrays to your objective function (which aren’t hashable for standard caches), you’ll need a quick wrapper to convert them to tuples first. Here’s a concrete example:

from functools import lru_cache
import numpy as np
from scipy.optimize import minimize

# Your slow, expensive function
def expensive_objective(x):
    # Simulate a long computation (replace with your actual function)
    import time
    time.sleep(1)
    return np.sum(x ** 2)

# Cache wrapper: convert numpy arrays to hashable tuples
@lru_cache(maxsize=None)
def cached_objective(x_tuple):
    x = np.array(x_tuple)
    return expensive_objective(x)

# Final objective for minimize: handles numpy arrays
def objective(x):
    return cached_objective(tuple(x))

# Example bound constraints
bounds = [(-5, 5) for _ in range(3)]

# Initialize with one of your precomputed points (pick the one with the lowest known value!)
precomputed_points = np.array([[1, 2, 3], [-2, -1, 0], [4, 1, -2]])
best_initial_point = precomputed_points[np.argmin([cached_objective(tuple(p)) for p in precomputed_points])]

# Run the optimization
result = minimize(objective, x0=best_initial_point, bounds=bounds, method="L-BFGS-B")

This ensures every unique point (whether from your precomputed set or the optimizer’s iterations) is only calculated once.

Optimizers perform best when they start close to the true minimum. Use your precomputed data to give the optimizer a head start:

• Pick the best initial point: Evaluate all your precomputed points, then use the one with the smallest function value as x0. This cuts down the number of iterations the optimizer needs to converge.
• Incremental warm-start: For methods like SLSQP or L-BFGS-B, you can run the optimizer in stages. Start with your best precomputed point, get a result, then use that result’s x as the new x0 for another run. Since you’re caching function values, you won’t re-calculate any points from the first run.

If your function is really slow (think seconds or minutes per evaluation), surrogate modeling is a game-changer. The idea is to train a cheap-to-evaluate "surrogate" model (like a radial basis function or Gaussian process) on your precomputed points, optimize the surrogate to find a candidate minimum, then evaluate the real function at that candidate to update the surrogate. Repeat until the candidate stops improving.

Here’s a simplified example using scipy’s RBF interpolator:

from scipy.interpolate import RBFInterpolator
import numpy as np
from scipy.optimize import minimize

# Your precomputed data (replace with your actual points/values)
known_x = np.array([[1, 2, 3], [-2, -1, 0], [4, 1, -2]])
known_y = np.array([expensive_objective(x) for x in known_x])

# Initialize the surrogate model
surrogate = RBFInterpolator(known_x, known_y)

def surrogate_objective(x):
    # RBFInterpolator expects 2D input, so reshape the 1D x from minimize
    return surrogate(x.reshape(1, -1))[0]

# Bound constraints (same as before)
bounds = [(-5, 5) for _ in range(3)]

# Iterate: optimize surrogate, update with real function value
tolerance = 1e-6
previous_best = np.inf

for _ in range(5):  # Adjust number of iterations based on your needs
    # Optimize the surrogate
    surrogate_result = minimize(surrogate_objective, x0=known_x[np.argmin(known_y)], bounds=bounds, method="L-BFGS-B")
    
    # Evaluate real function at the surrogate's minimum
    real_value = expensive_objective(surrogate_result.x)
    
    # Check if we've converged
    if abs(real_value - previous_best) < tolerance:
        break
    
    # Update the surrogate with the new point
    known_x = np.vstack([known_x, surrogate_result.x])
    known_y = np.hstack([known_y, real_value])
    surrogate = RBFInterpolator(known_x, known_y)
    previous_best = real_value

# Final result is the last real evaluation
final_result = {"x": surrogate_result.x, "fun": real_value}

This drastically reduces the number of times you call your expensive function, since most iterations work on the cheap surrogate.

Not all optimization methods are created equal for expensive functions. Stick to methods that minimize function evaluations, especially with bound constraints:

• L-BFGS-B: The go-to for bound-constrained problems. It’s efficient, requires only function evaluations (no gradients, though you can provide them if you have a cheap way to compute them), and converges quickly with fewer calls.
• SLSQP: Good if you have additional equality/inequality constraints, but may require more function evaluations than L-BFGS-B.
• Avoid Nelder-Mead or Powell: These methods use a lot of function evaluations to explore the space, which is terrible for slow functions.

内容的提问来源于stack exchange，提问作者Volodimir Kopey