Python NumPy带位移符合计数代码加速优化求助
问题背景
我此前曾发布过提升代码运行效率的相关问题,对应代码的核心功能是计算两个存储双通道时间值的NumPy数组之间的平均「符合计数(coincidences)」:当给定时间区间内两个通道均存在至少一个时间值时,即判定为一次符合事件。
我会先将长数组拆分为长度约10k-100k的分块(chunks),对分块做分箱(binned)处理后,配对计算对应分箱间的符合数:即数组1第1个分块的第n号分箱,与数组2第1个分块的第n号分箱配对统计。参考此前社区给出的优化建议修改代码后,运行速度已有显著提升。
目前我需要新增位移符合计数的计算逻辑:分别计算两个分块中第n号分箱与第n+shift号分箱间的符合数,遍历多个不同的shift取值完成全部计算。修改后的代码虽然比初始版本更快,但面对待分析的庞大数据量仍然运行过慢。
当前实现代码如下:
from numba import njit import numpy as np import matplotlib.pyplot as plt @njit def numba_histogram(a, b, n): hista, histb = np.zeros(n, dtype=np.intp), np.zeros(n, dtype=np.intp) a_min, a_max = min(a[0], b[0]), max(a[-1], b[-1]) for x, y in zip(a, b): bin = n * (x - a_min) / (a_max - a_min) if x == a_max: hista[n - 1] += 1 elif bin >= 0 and bin < n: hista[int(bin)] += 1 bin = n * (y - a_min) / (a_max - a_min) if y == a_max: histb[n - 1] += 1 elif bin >= 0 and bin < n: histb[int(bin)] += 1 return hista, histb @njit def calc_coincidence(t1,t2, NumChunks, NumofBins, td): CoincAllChunks = 0 last = 0.0 avg = len(t1) / NumChunks while last < len(t1): t1chunks = t1[int(last):int(last + avg)] t2chunks = t2[int(last):int(last + avg)] if (len(t1chunks)<=1) or (len(t2chunks)<=1): last += avg else: Hist1, Hist2 = numba_histogram(t1chunks,t2chunks,NumofBins) Hist2 = Hist2[td:] appendthis = np.zeros(td) Hist2 = np.append(Hist2, appendthis) CoincOneChunk = np.sum( (Hist1 > 0) * (Hist2 > 0) ) CoincAllChunks += CoincOneChunk last += avg return CoincAllChunks ##### 参数配置:分块数、分箱大小、位移数量 ######## NumChunks = 10 HowManyUnitsofns = 100 BinSize = HowManyUnitsofns*10**(-9) NumberofDelays= 10 ################################################### NumberofFiles = 10 CoinAllDelaysAllFiles = np.zeros(NumberofDelays, dtype=np.int64) ListofCoinforDiffDelays = [] for n in range(NumberofFiles): specificfile = 'file{}.h5'.format(n) t1, t2, DataTime = ReadFile(specificfile) #################### 归一化计算 ################ ChunkSize = (DataTime)/(NumChunks) NumofBins = int(ChunkSize/BinSize) Singles1 = len(t1) Singles2 = len(t2) Normalization = (Singles1*Singles2*BinSize)/DataTime #################################################### CoinAllDelaysOneFile = [] for i in range(NumberofDelays): CoinOneDelay = 0 NormCoinOneDelay = (calc_coincidence(t1,t2, NumChunks, NumofBins,i))/Normalization CoinAllDelaysOneFile = np.append(CoinAllDelaysOneFile, NormCoinOneDelay) CoinAllDelaysAllFiles = np.vstack((CoinAllDelaysAllFiles, CoinAllDelaysOneFile)) for k in range(NumberofDelays): coineachdelay = np.sum(CoinAllDelaysAllFiles[:,k]) ListofCoinforDiffDelays = np.append(ListofCoinforDiffDelays, coineachdelay) NormListofCoinforDiffDelays = ListofCoinforDiffDelays/(NumberofFiles) print(NormListofCoinforDiffDelays) tau = HowManyUnitsofns*np.arange(NumberofDelays) plt.scatter(tau,NormListofCoinforDiffDelays) plt.ylim(0,2)
补充说明
- 核心目标:观测符合计数随位移值的变化规律,计算每个位移对应的符合数,最终绘制符合数-位移值关系曲线
- 已准备对应测试数据可用于性能验证
求助内容
请提供可行的代码优化方向,进一步提升这段代码的运行速度,所有优化建议都欢迎。
内容的提问来源于stack exchange,提问作者Sonali Gera
相关产品推荐
相关产品推荐

