You need to enable JavaScript to run this app.
优惠活动
大模型
产品
解决方案
定价
更多

使用asyncio并行运行协程却同步执行,如何实现真正并行?

问题分析

asyncio 是单线程异步框架,协程切换的核心前提是当前协程遇到可等待的异步操作并执行await,此时事件循环才会主动切换到其他协程。但你的func和func1是纯CPU密集的同步函数,执行时会持续占用线程资源,不会触发协程切换,导致两个协程只能串行执行,总耗时为两个任务耗时之和。

解决方案

要实现真正的并行执行,需要把阻塞的CPU密集任务从asyncio的单线程事件循环中剥离,放到线程池或进程池中执行,让事件循环可以在等待任务完成时切换到其他协程。

方案1:使用asyncio.to_thread(Python 3.9+)

将同步函数放到线程池执行,代码简洁,适合轻度CPU密集或IO密集任务:

import asyncio
import time

def func():
    print("into the func")
    s = time.time()
    l = []
    for i in range(100000000):
        l.append(i)
    print("time in func is ", time.time() - s)

def func1():
    print("into the func1")
    s = time.time()
    l = []
    for i in range(100000000):
        l.append(i)
    print("time in func1 is ", time.time() - s)

async def exp():
    print("exp started")
    await asyncio.sleep(0.01)
    # 把同步函数提交到线程池执行,await等待结果
    await asyncio.to_thread(func)
    print("finished exp")
    return [1,2]

async def data():
    print("data started")
    await asyncio.sleep(0.01)
    await asyncio.to_thread(func1)
    print("finished data")
    return {'a':1}

async def start_coros():
    tasks = [asyncio.create_task(coro) for coro in (exp(), data())]
    await asyncio.wait(tasks)

# 统计总耗时
start_total = time.time()
asyncio.run(start_coros())
print("total time :", time.time() - start_total)

方案2:使用concurrent.futures.ProcessPoolExecutor

由于Python的GIL(全局解释器锁)限制了单线程CPU利用率,重度CPU密集任务更适合用进程池,能充分利用多核CPU,总耗时更接近单个任务的执行时间:

import asyncio
import time
from concurrent.futures import ProcessPoolExecutor

def func():
    print("into the func")
    s = time.time()
    l = []
    for i in range(100000000):
        l.append(i)
    print("time in func is ", time.time() - s)

def func1():
    print("into the func1")
    s = time.time()
    l = []
    for i in range(100000000):
        l.append(i)
    print("time in func1 is ", time.time() - s)

async def exp(executor):
    print("exp started")
    await asyncio.sleep(0.01)
    # 提交任务到进程池执行
    await asyncio.get_event_loop().run_in_executor(executor, func)
    print("finished exp")
    return [1,2]

async def data(executor):
    print("data started")
    await asyncio.sleep(0.01)
    await asyncio.get_event_loop().run_in_executor(executor, func1)
    print("finished data")
    return {'a':1}

async def start_coros():
    # 创建进程池,max_workers建议设为CPU核心数
    with ProcessPoolExecutor(max_workers=2) as executor:
        tasks = [
            asyncio.create_task(exp(executor)),
            asyncio.create_task(data(executor))
        ]
        await asyncio.wait(tasks)

start_total = time.time()
asyncio.run(start_coros())
print("total time :", time.time() - start_total)

内容的提问来源于stack exchange,提问作者coder

相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级,最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分), 限时 9.9 元起

最近更新时间:2026.08.24 06:06:07