关于concurrent.futures.Executor上下文管理器阻塞特性及不同使用方式差异的技术咨询
关于concurrent.futures.Executor上下文管理器阻塞特性及不同使用方式差异的技术咨询
我最近在使用concurrent.futures里的Executor上下文管理器时发现了一个让我困惑的现象,想请教大家:
- 当分开使用两个独立的
with语句创建ThreadPoolExecutor和ProcessPoolExecutor时,第二个Executor的任务必须等到第一个Executor的所有任务完成后才会启动 - 但如果把两个Executor放在同一个复合
with语句里,第二个Executor的任务可以和第一个的任务并行执行
我原本以为executor.submit()会立即返回Future对象,不会阻塞主线程,但看起来问题出在上下文管理器本身的行为上。想确认这个现象是否属实,有没有官方文档提到这种独立上下文管理器和复合上下文管理器的行为差异?
独立上下文管理器示例
import time from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor def task(name, delay): print(f"{time.time():.3f} {name} started") time.sleep(delay) print(f"{time.time():.3f} {name} finished") if __name__ == "__main__": print(f"{time.time():.3f} Main started") with ThreadPoolExecutor(max_workers=1) as thread_pool: thread_pool.submit(task, "Thread Task", 0.2) with ProcessPoolExecutor(max_workers=1) as process_pool: process_pool.submit(task, "Process Task", 0.1) print(f"{time.time():.3f} Main ended")
输出结果:
1743068624.365 Main started 1743068624.365 Thread Task started 1743068624.566 Thread Task finished 1743068624.571 Process Task started 1743068624.671 Process Task finished 1743068624.673 Main ended
可以看到,线程任务必须完全结束后,进程任务才会开始,我多次运行代码都是这个结果,没有出现并行的情况。
复合上下文管理器示例
import time from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor def task(name, delay): print(f"{time.time():.3f} {name} started") time.sleep(delay) print(f"{time.time():.3f} {name} finished") if __name__ == "__main__": print(f"{time.time():.3f} Main started") with ThreadPoolExecutor(max_workers=1) as thread_pool, ProcessPoolExecutor(max_workers=1) as process_pool: thread_pool.submit(task, "Thread Task", 0.2) process_pool.submit(task, "Process Task", 0.1) print(f"{time.time():.3f} Main ended")
输出结果:
1743068722.440 Main started 1743068722.441 Thread Task started 1743068722.443 Process Task started 1743068722.544 Process Task finished 1743068722.641 Thread Task finished 1743068722.641 Main ended
这里两个任务是并行执行的,进程任务甚至在较短延迟后先于线程任务完成,不管我调整延迟比例,复合上下文管理器下它们都会并行启动。
想请大家帮忙看看我对这个现象的解读有没有错误?
备注:内容来源于stack exchange,提问作者Han Qi
相关产品推荐
相关产品推荐

