Scipy differential_evolution在类中调用时运行速度大幅变慢的问题求助
Scipy differential_evolution在类中调用时运行速度大幅变慢的问题求助
我最近碰到一个挺让人困惑的性能问题,想请教下各位大佬。为了加速scipy.optimize.differential_evolution的计算,我因为worker=-1会遇到本地对象无法pickle的问题,所以改用了pathos.multiprocessing.ProcessingPool的map方法来做并行。
在简化的测试例子里,两种实现方式(普通函数式、类封装式)的耗时差异不大,但放到实际项目中时,类封装的方式总耗时居然是函数式的10倍以上!观察CPU负载的话,函数式实现能把所有核心跑满,而类封装的方式基本只用到一个核心,同时通过disp=True看到的函数评估输出速度也慢了很多。
下面是我用来复现的简化测试代码,以及实际项目里的核心代码片段:
测试复现代码
from os import cpu_count from time import time import numpy as np from pathos.multiprocessing import ProcessingPool as Pool from scipy.optimize import differential_evolution from scipy.integrate import quad from scipy.interpolate import CubicSpline atol=1e-13 rtol=1e-10 T = 10 bounds = [(-T,T)] # --- Approach 1 --- # x = np.linspace(0,10*2*np.pi,100) y = np.cos(x) cs = CubicSpline(x, y) def f(t): return cs(t)**2 ctime = time() def cost(t): t = t[0] q = 0 for _ in range(50): q += quad(lambda s: f(s*t), -T, T)[0] return q result = differential_evolution(cost, bounds, atol=atol, tol=rtol, updating='deferred', workers=Pool(cpu_count()).map) print("Approach 1:", time()-ctime) # --- Approach 2 --- # class A(): def define_f(self): x = np.linspace(0,10*2*np.pi,100) y = np.cos(x) cs = CubicSpline(x, y) def f(t): return cs(t)**2 return f f = A().define_f() class B(): def solve(self, f): def cost(t): t = t[0] q = 0 for _ in range(50): q += quad(lambda s: f(s*t), -T, T)[0] return q ctime = time() result = differential_evolution(cost, bounds, atol=atol, tol=rtol, updating='deferred', workers=Pool(cpu_count()).map) print("Approach 2:", time()-ctime) B().solve(f=f)
测试输出大概是这样:
Approach 1: 4.710453510284424 Approach 2: 5.638171672821045
但在实际项目里,第二种方式的耗时会是第一种的10倍以上。
实际项目核心代码片段
# Define fundamental solution. def W(t: np.ndarray) -> np.ndarray: if type(t) is np.ndarray: return np.stack([y0.sol(t).T, y1.sol(t).T], axis=-1).swapaxes(1,2) else: return np.array([y0.sol(t), y1.sol(t)]) # Define monodromy matrix and related quantities. C = W(period) C1 = np.eye(C.shape[0]) - C C2 = np.linalg.solve(C, C1) # Define cost function. def cost(t): if len(t.shape) > 1: t = np.reshape(t, (t.shape[1],)) B = W(t) def H(s: float, t: np.ndarray=t, B: np.ndarray=B): mask_s_le_t = s <= t W_s = W(s) X0 = np.linalg.solve(C1 @ W_s, B) X1 = np.linalg.solve(C2 @ W_s, B) return np.where(mask_s_le_t[:, None, None], X0, X1) return -quad_vec(lambda s: np.sum(H(s)**2, axis=(1,2)), 0, period)[0] # Maximize. maxi = -minimize(cost, bounds=[(0,period)], atol=atol, tol=rtol, updating='deferred', vectorized=1).fun
有没有大佬能帮我分析下,为什么类封装的方式会导致并行失效、性能暴跌呢?感谢各位的指点!
备注:内容来源于stack exchange,提问作者Hannes
相关产品推荐
相关产品推荐

