Jupyter中使用Multiprocessing.Pool计算DNA序列比对Shannon熵时无限运行无返回的问题求助
Jupyter中使用Multiprocessing.Pool计算DNA序列比对Shannon熵时无限运行无返回的问题求助
我最近在尝试给计算DNA比对序列每个位点香农熵的函数加上多进程来提速,但程序一直无限运行,既不返回结果也不报错,实在搞不懂问题出在哪。
具体来说,我想把Multiprocessing.Pool整合到一个可以处理任意比对文件的熵计算类里,但听说不建议在类的__init__方法里用if __name__ == '__main__':,不确定这样直接整合到底行不行。我在Jupyter环境里运行过,也试过在Jupyter外的Python 3.8.5环境,结果都一样。
当我运行下面的代码时,串行计算能正常返回熵数组,但并行计算的部分就会无限跑下去,没有任何返回或报错。
我是多进程新手,真的非常需要大家的帮助!另外想请教下,是不是没办法直接把多进程整合到类对象里,必须用子进程调用单独脚本才行?
from Bio import AlignIO from multiprocessing import Pool from scipy.stats import entropy import pandas as pd from time import perf_counter path = 'path_to_aln_fasta' alignment = AlignIO.read(path, 'fasta') def calc_entropy(column): series = pd.Series(column) vals = series.value_counts(normalize=False) return entropy(vals) def pool_calc_entropy(alignment): pool = Pool(8) entropies = pool.map(calc_entropy, [alignment[:, i] for i in range(alignment.get_alignment_length())]) return entropies start = perf_counter() entropies_series = [] for i in range(alignment.get_alignment_length()): column = alignment[:, i] entropies_series.append(calc_entropy(column)) end = perf_counter() series_time = end-start print(f'series time: {series_time}') start = perf_counter() entropies_pool = pool_calc_entropy(alignment) end = perf_counter() parallel_time = end-start print(f'parallel time: {parallel_time}') print(f'pool time is {round((parallel_time)/(series_time), 2)} times faster than series time')
输入示例path_to_aln_fasta的内容:
>record_one GGTTTGATGTATATGCTCATTACTTATTTGGATTTTTGATAGAAATAAAACAGCTTTAAGCGCTTCATTACAAGTCACGCAGTTACACCTGTATCGCAGTGTTTACCATCTTGAGGGTAATTTTGCATTAGTGGACAGCTAGACTCAGCCACGCCTATACCCTTCGGAACATCGAAACATACCAACCATGATATCGTCGA >record_two ATGACTACTCGGTTACTTCTAATTGTTGCTAGCTGTACTTTGTGAAGTGAACAATAAGGTTTTAAAACCGTTAGACACAGTGCCTCTTTTAGGAAGATTCTTGATAATTTGCCCTGGTTTATGCATCGTTTGCACAGATTGACTTAAAGACTACGTCATAATAACTGCGCCTTGACTGCGGATTGAGCAATGCCTGCACA >record_three AGTAAATTTTATAGAGGTATTGCGCACCCTCGGAACTTTACAAAGCACAACCTGGATCACACTATTTCCGAGACCCACCTGACTGTGTTGTCGAGTTATCGATCCTATATATTTAAAGTTGGTTTTCAAAACATTTTTCGTTTGCGTTGTATACGGAGCCGTAGATAAGCGCATTTCGGTCATCAATCAGGGACAAATTA >record_four TTGAAATTAACATGCAACCTGACATGGTTTATTCTGCGGCATACAGAGAAATTGTGTTGATTGAACTGATTAGGATACTCCGGTCCCGTGTTTGAATTGTGCTAAAACCTGGTTAGAGATCGCGTCAATTCTCCATCCAGAAGAAGAACAATTTTAGGACCCGTTGTCATTTCCCTTAAAGGTTTTTGTAACAGTCCGAT
备注:内容来源于stack exchange,提问作者darkwing
相关产品推荐
相关产品推荐

