You need to enable JavaScript to run this app. Python For循环生成衰减权重的高性能优化实现方案
优惠活动
大模型
产品
解决方案
定价
更多

Python For循环生成衰减权重的高性能优化实现方案

批量生成衰减权重的性能优化

我需要批量生成符合指定规则的衰减权重,现有Python实现逻辑符合预期,但其中的For循环执行效率极低,本地测试单循环运行耗时达到1分05秒。

原始实现代码

# Imports
from sympy.solvers.solveset import solveset_real
from sympy import Symbol, Eq

# Define variables
initial_value = 1
rate = Symbol('r')
decay_obs_window = 1480346
target_decay = .15

# Solver to calculate decay rate
decay_rate = solveset_real(Eq((initial_value - rate * decay_obs_window), target_decay), rate).args[0]

# Generate weights
weights = []
for i in range(5723673):
    # How to handle data BEYOND decay_obs_window
    if i > decay_obs_window and target_decay == 0:
        # Record a weight of zero
        weights.append(0)
    elif i > decay_obs_window and target_decay > 0:
        # Record the final target weight
        weights.append(decayed_weight)
    # How to handle data WITHIN decay_obs_window
    else:
        # Calculate the new slightly decayed weight
        decayed_weight = 1 - (decay_rate * i)
        weights.append(decayed_weight)

weights[0:10]

我尝试改用列表推导式优化执行效率,代码可正常运行,但性能无明显提升,优化后代码如下:

weights = [0 if i > decay_obs_window and target_decay == 0 else decayed_weight if i > decay_obs_window and target_decay > 0 else (decayed_weight := 1 - (decay_rate * i)) for i in range(len(weights_df))]

请问有什么更高效的实现方案可以大幅降低运行耗时?

最终优化方案

最终采用了基于numpy向量化运算的优化方案,本地测试总运行耗时仅425ms,代码如下:

import numpy as np
from sympy.solvers.solveset import solveset_real
from sympy import Symbol, Eq

# Define variables
initial_value = 1
rate = Symbol('r')
decay_obs_window = 1480346
target_decay = .15

# Instantiate weights array
weights = np.zeros(5723673)

# Solver to calculate decay rate
decay_rate = solveset_real(Eq((initial_value - rate * decay_obs_window), target_decay), rate).args[0]

# Fix a bug where numpy doesn't like sympy floats :(
decay_rate = float(decay_rate)

# How to weight observations WITHIN decay_obs_window
weights[:decay_obs_window + 1] = 1 - np.arange(decay_obs_window + 1) * decay_rate

# How to weight observations BEYOND decay_obs_window
weights[decay_obs_window + 1 : 5723673] = target_decay

weights

优化核心逻辑

  • 抛弃逐元素遍历的Python原生循环,使用Numpy的向量化运算,底层由C实现,大幅提升大规模数值计算效率
  • 直接按区间批量赋值,不需要对每个元素做分支判断,进一步减少运算开销

内容的提问来源于stack exchange,提问作者matteblack

相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级,最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分), 限时 9.9 元起

最近更新时间:2026.10.17 06:15:04