Celery+Eventlet启动Worker遇NotImplementedError：unsupported platform

环境与操作 #

• 技术栈：Celery、Eventlet、django-celery-beat
• 启动命令：celery -A app worker -P eventlet -c 100 -l info --logfile=celery.log
• 依赖安装：通过miniconda的conda-forge通道安装Python3.10及相关组件

完整报错栈 #

(sample_app) neerajgoyal@Neerajs-MacBook-Pro app % celery -A app worker -P eventlet -c 100 -l info  --logfile=celery.log
Traceback (most recent call last):
  File "/Users/neerajgoyal/miniconda3/envs/sample_app/bin/celery", line 10, in <module>
    sys.exit(main())
  File "/Users/neerajgoyal/miniconda3/envs/sample_app/lib/python3.10/site-packages/celery/__main__.py", line 13, in main
    maybe_patch_concurrency()
  File "/Users/neerajgoyal/miniconda3/envs/sample_app/lib/python3.10/site-packages/celery/__init__.py", line 140, in maybe_patch_concurrency
    patcher()
  File "/Users/neerajgoyal/miniconda3/envs/sample_app/lib/python3.10/site-packages/celery/__init__.py", line 101, in _patch_eventlet
    import eventlet.debug
  File "/Users/neerajgoyal/miniconda3/envs/sample_app/lib/python3.10/site-packages/eventlet/__init__.py", line 17, in <module>
    from eventlet import convenience
  File "/Users/neerajgoyal/miniconda3/envs/sample_app/lib/python3.10/site-packages/eventlet/convenience.py", line 7, in <module>
    from eventlet.green import socket
  File "/Users/neerajgoyal/miniconda3/envs/sample_app/lib/python3.10/site-packages/eventlet/green/socket.py", line 21, in <module>
    from eventlet.support import greendns
  File "/Users/neerajgoyal/miniconda3/envs/sample_app/lib/python3.10/site-packages/eventlet/support/greendns.py", line 79, in <module>
    setattr(dns, pkg, import_patched('dns.' + pkg))
  File "/Users/neerajgoyal/miniconda3/envs/sample_app/lib/python3.10/site-packages/eventlet/support/greendns.py", line 61, in import_patched
    return patcher.import_patched(module_name, **modules)
  File "/Users/neerajgoyal/miniconda3/envs/sample_app/lib/python3.10/site-packages/eventlet/patcher.py", line 132, in import_patched
    return inject(
  File "/Users/neerajgoyal/miniconda3/envs/sample_app/lib/python3.10/site-packages/eventlet/patcher.py", line 109, in inject
    module = __import__(module_name, {}, {}, module_name.split('.')[:-1])
  File "/Users/neerajgoyal/miniconda3/envs/sample_app/lib/python3.10/site-packages/dns/asyncquery.py", line 38, in <module>
    from dns.query import (
  File "/Users/neerajgoyal/miniconda3/envs/sample_app/lib/python3.10/site-packages/dns/query.py", line 63, in <module>
    import httpcore
  File "/Users/neerajgoyal/miniconda3/envs/sample_app/lib/python3.10/site-packages/httpcore/__init__.py", line 1, in <module>
    from ._api import request, stream
  File "/Users/neerajgoyal/miniconda3/envs/sample_app/lib/python3.10/site-packages/httpcore/_api.py", line 5, in <module>
    from ._sync.connection_pool import ConnectionPool
  File "/Users/neerajgoyal/miniconda3/envs/sample_app/lib/python3.10/site-packages/httpcore/_sync/__init__.py", line 1, in <module>
    from .connection import HTTPConnection
  File "/Users/neerajgoyal/miniconda3/envs/sample_app/lib/python3.10/site-packages/httpcore/_sync/connection.py", line 12, in <module>
    from .._synchronization import Lock
  File "/Users/neerajgoyal/miniconda3/envs/sample_app/lib/python3.10/site-packages/httpcore/_synchronization.py", line 13, in <module>
    import trio
  File "/Users/neerajgoyal/miniconda3/envs/sample_app/lib/python3.10/site-packages/trio/__init__.py", line 19, in <module>
    from ._core import TASK_STATUS_IGNORED as TASK_STATUS_IGNORED  # isort: skip
  File "/Users/neerajgoyal/miniconda3/envs/sample_app/lib/python3.10/site-packages/trio/_core/__init__.py", line 21, in <module>
    from ._local import RunVar
  File "/Users/neerajgoyal/miniconda3/envs/sample_app/lib/python3.10/site-packages/trio/_core/_local.py", line 5, in <module>
    from . import _run
  File "/Users/neerajgoyal/miniconda3/envs/sample_app/lib/python3.10/site-packages/trio/_core/_run.py", line 2548, in <module>
    raise NotImplementedError("unsupported platform")
NotImplementedError: unsupported platform

排查与解决方案 #

从报错链可见：Eventlet的greendns模块触发了dnspython的加载，新版dnspython引入了httpcore依赖，而httpcore依赖的trio库在当前Mac环境下抛出平台不支持错误。

方案1：降级dnspython版本 #

Eventlet对dnspython 2.3.0+版本兼容性不佳，降级到2.2.x版本可解决依赖链问题：

conda install -c conda-forge dnspython=2.2.1

方案2：禁用Eventlet的DNS补丁 #

在项目的Celery配置文件（如celery.py）中添加以下代码，跳过Eventlet对DNS模块的猴子补丁：

import eventlet
# 禁用DNS补丁，避免触发有问题的依赖链
eventlet.monkey_patch(dns=False)

添加后重新执行启动命令。

方案3：替换Eventlet为Gevent #

如果前两种方案无效，可改用Gevent作为Celery的并发池：

1. 安装Gevent：

conda install -c conda-forge gevent

2. 修改启动命令：

celery -A app worker -P gevent -c 100 -l info --logfile=celery.log

内容的提问来源于stack exchange，提问作者neerajgoyal12