Scipy 1.9.x中Beta-Binomial分布拟合异常:是否存在破坏性变更?
Scipy 1.9.0升级后Beta-Binomial分布拟合失败问题
将Scipy版本从1.8.1升级至1.9.0,保持Numpy版本为1.24.4后,项目运行出现异常。以下代码可复现该问题,场景为使用Scipy的Beta-Binomial分布拟合概率分布:
from collections import Counter import numpy as np from scipy import stats from scipy.optimize import differential_evolution def func_nll(free_params, *args): """Negative log likelihood used in the optimizer to fit the best probability distribution available. More information on the optimizer used in https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.differential_evolution.html Arguments: free_params: probability distribution parameters under tuning. Depending on the probability distribution to fit, there can be one or more *args: custom distribution arguments, including the Returns: Negative log likelihood value used in the optimizer """ dist, x, shift = args # negative log-likelihood function negativeLogLikelihood = -dist.logpmf(x, loc=shift, *free_params).sum() if np.isnan(negativeLogLikelihood): # occurs when x is outside of support negativeLogLikelihood = np.inf # we don't want that return negativeLogLikelihood def fit(x, shift): bounds = [(0, n+1), (1, 700), (1, 700)] dist = stats.betabinom opt_res = differential_evolution(func_nll, bounds, maxiter=100, tol=0.001, seed=45, args=(dist, x, shift)) return opt_res x = [28, 67, 69, 69, 71, 69, 55, 65, 68, 67, 67, 68, 69, 60, 40] n = max(x) mode_value = Counter(x).most_common(1)[0][1] n_zero = 4288 # high number of zeroes is_zero_inflated = (n_zero >= mode_value) shift_loc = 0 if is_zero_inflated: shift_loc = 1 b_param = fit(x, shift=shift_loc) print(b_param)
在scipy==1.8.1环境下,代码输出:
df = fun(x) - f0 fun: 49.59657327021416 message: 'Optimization terminated successfully.' nfev: 2802 nit: 61 success: True x: array([70.00205357, 6.28931939, 1.0005225 ])
升级至scipy==1.9.0后,代码输出:
df = fun(x) - f0 fun: inf message: 'Maximum number of iterations has been exceeded.' nfev: 9129 nit: 100 success: False x: array([ 45.83055307, 3.17350763, 596.7724981 ])
想了解Scipy 1.9.x中Beta-Binomial分布的实现是否存在破坏性变更,导致该问题出现。
内容的提问来源于stack exchange,提问作者2m4t
相关产品推荐
相关产品推荐

