Python异步函数无法同时启动,求原因及解决办法
问题描述
我编写了如下Python异步程序:
import datetime import asyncio import time import math async def count1(): s = 0 print('start time count 1: ' +str(datetime.datetime.now())) for i in range(100000000): s += math.cos(i) print('end time count 1: ' +str(datetime.datetime.now())) return s async def count2(): s = 0 print('start time count 2: ' +str(datetime.datetime.now())) for i in range(1000000): s += math.cos(i) print('end time count 2: ' +str(datetime.datetime.now())) return s async def main(): start_time = time.time() task = asyncio.gather(count1(), count2()) results = await task end_time = time.time() print(f"Result 1: {results[0]}") print(f"Result 2: {results[1]}") print(f"Total time taken: {end_time - start_time:.2f} seconds") asyncio.run(main())
程序运行输出如下:
start time count 1: 2023-02-16 12:26:19.322523 end time count 1: 2023-02-16 12:26:40.866866 start time count 2: 2023-02-16 12:26:40.868166 end time count 2: 2023-02-16 12:26:41.055005 Result 1: 1.534369444774577 Result 2: -0.28870546796843 Total time taken: 21.73 seconds
我期望count1()和count2()能同时启动执行,但从输出可见count2()仅在count1()结束后才开始运行。我还尝试将main()中的代码替换为:
task1 = asyncio.create_task(count1()) task2 = asyncio.create_task(count2()) result1 = await task1 result2 = await task2
但依然无法实现两函数同时启动,请问这是为什么?
解答
这是因为你的count1和count2函数里都是纯CPU密集型的同步代码,没有任何await调用——而Python的asyncio是基于协作式多任务的,只有当任务主动通过await让出CPU控制权时,事件循环才会切换到其他任务执行。
在你的代码里,count1启动后,会一直执行那个1亿次的循环,全程没有触发任何await操作,事件循环根本没机会切换到count2任务,只能等count1完全执行完,才会去运行count2。
如果要让CPU密集型任务实现并行,你需要用以下方式:
- 使用
concurrent.futures.ProcessPoolExecutor把CPU密集型任务放到子进程中执行,这样能绕过GIL的限制,真正实现并行 - 或者在CPU密集型循环中,定期插入
await asyncio.sleep(0)来主动让出控制权,不过这种方式只是伪并行,总耗时不会减少,只是让两个任务交替执行
举个用ProcessPoolExecutor改造的例子:
import datetime import asyncio import time import math from concurrent.futures import ProcessPoolExecutor def count1(): s = 0 print('start time count 1: ' +str(datetime.datetime.now())) for i in range(100000000): s += math.cos(i) print('end time count 1: ' +str(datetime.datetime.now())) return s def count2(): s = 0 print('start time count 2: ' +str(datetime.datetime.now())) for i in range(1000000): s += math.cos(i) print('end time count 2: ' +str(datetime.datetime.now())) return s async def main(): start_time = time.time() executor = ProcessPoolExecutor() loop = asyncio.get_running_loop() # 把CPU密集型任务提交到进程池 task1 = loop.run_in_executor(executor, count1) task2 = loop.run_in_executor(executor, count2) results = await asyncio.gather(task1, task2) end_time = time.time() print(f"Result 1: {results[0]}") print(f"Result 2: {results[1]}") print(f"Total time taken: {end_time - start_time:.2f} seconds") executor.shutdown() asyncio.run(main())
这样改造后,count1和count2会在不同的子进程中并行执行,总耗时会接近两个任务中耗时较长的那个(而不是两者相加)。
内容的提问来源于stack exchange,提问作者Drbf1337
相关产品推荐
相关产品推荐

