使用Python 3.11时Gunicorn(gevent) Worker启动失败求助

环境信息 #

• 系统：Alpine 3.16.2
• Python版本：3.11.0
• pip版本：22.3
• 核心依赖版本：
  • gunicorn==20.1.0
  • gevent==22.10.2
  • greenlet==2.0.1
• 额外依赖：Django、谷歌相关库等

错误日志 #

启动Gunicorn gevent Worker时触发如下错误：

[2022-11-30 09:19:27 +0000] [6] [INFO] Starting gunicorn 20.1.0
[2022-11-30 09:19:27 +0000] [6] [INFO] Listening at: https://0.0.0.0:8443 (6)
[2022-11-30 09:19:27 +0000] [6] [INFO] Using worker: gevent
[2022-11-30 09:19:27 +0000] [7] [INFO] Booting worker with pid: 7
[2022-11-30 09:19:28 +0000] [8] [INFO] Booting worker with pid: 8
[2022-11-30 09:19:29 +0000] [7] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python3.11/site-packages/gunicorn/workers/ggevent.py", line 144, in init_process
    self.patch()
  File "/usr/local/lib/python3.11/site-packages/gunicorn/workers/ggevent.py", line 38, in patch
    monkey.patch_all()
  File "/usr/local/lib/python3.11/site-packages/gevent/monkey.py", line 1263, in patch_all
    patch_thread(Event=Event, _warnings=_warnings)
  File "/usr/local/lib/python3.11/site-packages/gevent/monkey.py", line 200, in ignores
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/gevent/monkey.py", line 791, in patch_thread
    _patch_existing_locks(threading_mod)
  File "/usr/local/lib/python3.11/site-packages/gevent/monkey.py", line 672, in _patch_existing_locks
    if isinstance(o, rlock_type):
       ^^^^^^^^^^^^^^^^^^^^^^^^^
**ReferenceError: weakly-referenced object no longer exists**
[2022-11-30 09:19:29 +0000] [7] [INFO] Worker exiting (pid: 7)
[2022-11-30 09:19:29 +0000] [8] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python3.11/site-packages/gunicorn/workers/ggevent.py", line 144, in init_process
    self.patch()
  File "/usr/local/lib/python3.11/site-packages/gunicorn/workers/ggevent.py", line 38, in patch
    monkey.patch_all()
  File "/usr/local/lib/python3.11/site-packages/gevent/monkey.py", line 1263, in patch_all
    patch_thread(Event=Event, _warnings=_warnings)
  File "/usr/local/lib/python3.11/site-packages/gevent/monkey.py", line 200, in ignores
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/gevent/monkey.py", line 791, in patch_thread
    _patch_existing_locks(threading_mod)
  File "/usr/local/lib/python3.11/site-packages/gevent/monkey.py", line 672, in _patch_existing_locks
    if isinstance(o, rlock_type):
       ^^^^^^^^^^^^^^^^^^^^^^^^^
**ReferenceError: weakly-referenced object no longer exists**
[2022-11-30 09:19:29 +0000] [8] [INFO] Worker exiting (pid: 8)
[2022-11-30 09:19:30 +0000] [6] [WARNING] Worker with pid 7 was terminated due to signal 15
[2022-11-30 09:19:30 +0000] [6] [INFO] Shutting down: Master
[2022-11-30 09:19:30 +0000] [6] [INFO] Reason: Worker failed to boot.

对比验证 #

相同Alpine 3.16.2环境下，使用Python 3.10.7 + pip 22.2.2搭配完全一致的依赖版本，Gunicorn gevent Worker可正常启动运行。

解决方案 #

1. 升级gevent到兼容Python 3.11的版本 #

Python 3.11对线程模块内部结构做了调整，gevent 22.10.2未适配该变化导致弱引用错误。升级到gevent 23.7.0及以上版本即可修复此问题：

pip install --upgrade gevent>=23.7.0

2. 手动控制猴子补丁时机（临时 workaround） #

若暂时无法升级gevent，可修改补丁执行逻辑：

• 在Django项目的wsgi.py文件最顶部添加：

  from gevent import monkey
  monkey.patch_all()
  

• 启动Gunicorn时禁用自带的猴子补丁：

  gunicorn --worker-class gevent --no-monkey-patch your_project.wsgi:application
  

排查思路 #

• 版本兼容性验证：确认gevent版本对Python 3.11的支持情况，Python 3.11的适配支持从gevent 23.1.0开始逐步完善。
• 补丁执行顺序排查：检查是否有第三方库在Gunicorn初始化worker前就已创建线程锁对象，导致gevent补丁处理时出现弱引用失效。
• 最小环境测试：搭建仅包含Python、gunicorn、gevent的极简环境，测试启动是否正常，再逐步添加其他依赖，定位是否存在依赖冲突。

内容的提问来源于stack exchange，提问作者Pavan Tatikonda