为何禁用GIL的Python 3.13.0b3性能远逊于Python 3.12.0?
无GIL Python 3.13性能测试与问题排查
我对编译时添加--disable-gil参数的Python 3.13.0b3和Python 3.12.0做了性能测试,通过ThreadPoolExecutor或ProcessPoolExecutor计算斐波那契数列。根据引入GIL禁用机制的PEP文档说明,该特性仅会带来约5-8%的性能开销,主要源于偏向引用计数和逐对象锁。但我的测试显示二者性能差异显著:禁用GIL的Python3.13虽能通过ThreadPoolExecutor利用全部CPU核心,但速度远慢于带GIL的Python3.12,耗时对应的时钟周期数是后者数倍。
测试代码
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor import datetime from functools import partial import sys import logging import multiprocessing logging.basicConfig( format='%(levelname)s: %(message)s', ) logger = logging.getLogger(__name__) logger.setLevel(logging.INFO) cpus = multiprocessing.cpu_count() pool_executor = ProcessPoolExecutor if len(sys.argv) > 1 and sys.argv[1] == '1' else ThreadPoolExecutor python_version_str = f'{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}' logger.info(f'Executor={pool_executor.__name__}, python={python_version_str}, cpus={cpus}') def fibonacci(n: int) -> int: if n < 0: raise ValueError("Incorrect input") elif n == 0: return 0 elif n == 1 or n == 2: return 1 else: return fibonacci(n-1) + fibonacci(n-2) start = datetime.datetime.now() with pool_executor(8) as executor: for task_id in range(30): executor.submit(partial(fibonacci, 30)) executor.shutdown(wait=True) end = datetime.datetime.now() elapsed = end - start logger.info(f'Elapsed: {elapsed.total_seconds():.2f} seconds')
初始测试结果(Linux 5.15.0-58-generic,Ubuntu 20.04.6 LTS)
# TEST Linux 5.15.0-58-generic, Ubuntu 20.04.6 LTS INFO: Executor=ThreadPoolExecutor, python=3.12.0, cpus=2 INFO: Elapsed: 10.54 seconds INFO: Executor=ProcessPoolExecutor, python=3.12.0, cpus=2 INFO: Elapsed: 4.33 seconds INFO: Executor=ThreadPoolExecutor, python=3.13.0b3, cpus=2 INFO: Elapsed: 22.48 seconds INFO: Executor=ProcessPoolExecutor, python=3.13.0b3, cpus=2 INFO: Elapsed: 22.03 seconds
问题
为何我的测试结果与pyperformance基准测试的开销差异如此巨大?
编辑1:补充测试
- 将线程池大小改为
pool_executor(cpus),测试结果仍无明显变化 - 参考视频及对应测试代码执行素数计算测试,结果显示Python3.13的单线程、多线程、多进程性能均远慢于Python3.12,而视频中的结果与PEP描述的开销相符。测试结果如下:
Version of python: 3.12.0a7 (main, Oct 8 2023, 12:41:37) [GCC 9.4.0] GIL cannot be disabled Single-threaded: 78498 primes in 6.67 seconds Threaded: 78498 primes in 7.89 seconds Multiprocessed: 78498 primes in 5.85 seconds Version of python: 3.13.0b3 experimental free-threading build (heads/3.13.0b3:7b413952e8, Jul 27 2024, 11:19:31) [GCC 9.4.0] GIL is disabled Single-threaded: 78498 primes in 61.42 seconds Threaded: 78498 primes in 32.29 seconds Multiprocessed: 78498 primes in 39.85 seconds
编辑2:优化编译参数后的性能改善
使用./configure --disable-gil --enable-optimizations参数重新编译Python3.13(此前使用./configure --with-pydebug --disable-gil编译),性能出现显著改善:
斐波那契基准测试结果
INFO: Executor=ThreadPoolExecutor, python=3.12.0, cpus=2 INFO: Elapsed: 10.25 seconds INFO: Executor=ProcessPoolExecutor, python=3.12.0, cpus=2 INFO: Elapsed: 4.27 seconds INFO: Executor=ThreadPoolExecutor, python=3.13.0, cpus=2 INFO: Elapsed: 6.94 seconds INFO: Executor=ProcessPoolExecutor, python=3.13.0, cpus=2 INFO: Elapsed: 6.94 seconds
素数基准测试结果
Version of python: 3.12.0a7 (main, Oct 8 2023, 12:41:37) [GCC 9.4.0] GIL cannot be disabled Single-threaded: 78498 primes in 5.77 seconds Threaded: 78498 primes in 7.21 seconds Multiprocessed: 78498 primes in 3.23 seconds Version of python: 3.13.0b3 experimental free-threading build (heads/3.13.0b3:7b413952e8, Aug 3 2024, 14:47:48) [GCC 9.4.0] GIL is disabled Single-threaded: 78498 primes in 7.99 seconds Threaded: 78498 primes in 4.17 seconds Multiprocessed: 78498 primes in 4.40 seconds
性能与资源分析
从Python3.12多进程切换到Python3.13无GIL多线程可大幅节省内存(仅需单个进程)。在2核机器上对比CPU开销:
- [斐波那契] Python3.13多线程相对Python3.12多进程:
(6.94-4.27)/4.27*100%≈63%开销 - [素数] Python3.13多线程相对Python3.12多进程:
(4.17-3.23)/3.23*100%≈29%开销
内容的提问来源于stack exchange,提问作者K4liber




