You need to enable JavaScript to run this app.
优惠活动
大模型
产品
解决方案
定价
更多
文档控制台
注册

为何禁用GIL的Python 3.13.0b3性能远逊于Python 3.12.0?

无GIL Python 3.13性能测试与问题排查

我对编译时添加--disable-gil参数的Python 3.13.0b3和Python 3.12.0做了性能测试,通过ThreadPoolExecutor或ProcessPoolExecutor计算斐波那契数列。根据引入GIL禁用机制的PEP文档说明,该特性仅会带来约5-8%的性能开销,主要源于偏向引用计数和逐对象锁。但我的测试显示二者性能差异显著:禁用GIL的Python3.13虽能通过ThreadPoolExecutor利用全部CPU核心,但速度远慢于带GIL的Python3.12,耗时对应的时钟周期数是后者数倍。

测试代码

from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor
import datetime
from functools import partial
import sys
import logging
import multiprocessing

logging.basicConfig(
    format='%(levelname)s: %(message)s',
)
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
cpus = multiprocessing.cpu_count()
pool_executor = ProcessPoolExecutor if len(sys.argv) > 1 and sys.argv[1] == '1' else ThreadPoolExecutor
python_version_str = f'{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}'
logger.info(f'Executor={pool_executor.__name__}, python={python_version_str}, cpus={cpus}')


def fibonacci(n: int) -> int:
    if n < 0:
        raise ValueError("Incorrect input")
    elif n == 0:
        return 0
    elif n == 1 or n == 2:
        return 1
    else:
        return fibonacci(n-1) + fibonacci(n-2)

start = datetime.datetime.now()

with pool_executor(8) as executor:
    for task_id in range(30):
        executor.submit(partial(fibonacci, 30))

    executor.shutdown(wait=True)

end = datetime.datetime.now()
elapsed = end - start
logger.info(f'Elapsed: {elapsed.total_seconds():.2f} seconds')

初始测试结果(Linux 5.15.0-58-generic,Ubuntu 20.04.6 LTS)

# TEST Linux 5.15.0-58-generic, Ubuntu 20.04.6 LTS

INFO: Executor=ThreadPoolExecutor, python=3.12.0, cpus=2
INFO: Elapsed: 10.54 seconds

INFO: Executor=ProcessPoolExecutor, python=3.12.0, cpus=2
INFO: Elapsed: 4.33 seconds

INFO: Executor=ThreadPoolExecutor, python=3.13.0b3, cpus=2
INFO: Elapsed: 22.48 seconds

INFO: Executor=ProcessPoolExecutor, python=3.13.0b3, cpus=2
INFO: Elapsed: 22.03 seconds

问题

为何我的测试结果与pyperformance基准测试的开销差异如此巨大?

编辑1:补充测试

  • 将线程池大小改为pool_executor(cpus),测试结果仍无明显变化
  • 参考视频及对应测试代码执行素数计算测试,结果显示Python3.13的单线程、多线程、多进程性能均远慢于Python3.12,而视频中的结果与PEP描述的开销相符。测试结果如下:
Version of python: 3.12.0a7 (main, Oct  8 2023, 12:41:37) [GCC 9.4.0]
GIL cannot be disabled
Single-threaded: 78498 primes in 6.67 seconds
Threaded: 78498 primes in 7.89 seconds
Multiprocessed: 78498 primes in 5.85 seconds

Version of python: 3.13.0b3 experimental free-threading build (heads/3.13.0b3:7b413952e8, Jul 27 2024, 11:19:31) [GCC 9.4.0]
GIL is disabled
Single-threaded: 78498 primes in 61.42 seconds
Threaded: 78498 primes in 32.29 seconds
Multiprocessed: 78498 primes in 39.85 seconds

编辑2:优化编译参数后的性能改善

使用./configure --disable-gil --enable-optimizations参数重新编译Python3.13(此前使用./configure --with-pydebug --disable-gil编译),性能出现显著改善:

斐波那契基准测试结果

INFO: Executor=ThreadPoolExecutor, python=3.12.0, cpus=2
INFO: Elapsed: 10.25 seconds

INFO: Executor=ProcessPoolExecutor, python=3.12.0, cpus=2
INFO: Elapsed: 4.27 seconds

INFO: Executor=ThreadPoolExecutor, python=3.13.0, cpus=2
INFO: Elapsed: 6.94 seconds

INFO: Executor=ProcessPoolExecutor, python=3.13.0, cpus=2
INFO: Elapsed: 6.94 seconds

素数基准测试结果

Version of python: 3.12.0a7 (main, Oct  8 2023, 12:41:37) [GCC 9.4.0]
GIL cannot be disabled
Single-threaded: 78498 primes in 5.77 seconds
Threaded: 78498 primes in 7.21 seconds
Multiprocessed: 78498 primes in 3.23 seconds

Version of python: 3.13.0b3 experimental free-threading build (heads/3.13.0b3:7b413952e8, Aug  3 2024, 14:47:48) [GCC 9.4.0]
GIL is disabled
Single-threaded: 78498 primes in 7.99 seconds
Threaded: 78498 primes in 4.17 seconds
Multiprocessed: 78498 primes in 4.40 seconds

性能与资源分析

从Python3.12多进程切换到Python3.13无GIL多线程可大幅节省内存(仅需单个进程)。在2核机器上对比CPU开销:

  • [斐波那契] Python3.13多线程相对Python3.12多进程:(6.94-4.27)/4.27*100%≈63%开销
  • [素数] Python3.13多线程相对Python3.12多进程:(4.17-3.23)/3.23*100%≈29%开销

内容的提问来源于stack exchange,提问作者K4liber

火山引擎 最新活动