导入Playwright库后无法启动Celery Worker的问题求助

问题详情 #

我尝试在Celery任务中执行一段简单的Playwright代码，任务代码如下：

@shared_task
def get_title():
    with sync_playwright() as playwright:
        browser = playwright.chromium.launch()
        page = browser.new_page()
        page.goto('https://github.com')
        print(page.title())
        browser.close()

启动Celery Worker时立即报错，我试过Playwright的同步、异步多种调用方式均无效，移除Playwright的导入及相关代码后，Celery Worker可正常启动。

我推测问题与导入顺序有关，尝试在Celery之前导入Playwright或select库，但仍出现类似错误：'select' has no attribute 'epoll'，最初还没有完整的回溯信息。

我的celery.py代码如下：

# celery.py
import os
from celery import Celery

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings.local")

app = Celery("celery_app")

app.config_from_object("django.conf:settings", namespace="CELERY")

app.autodiscover_tasks()

运行环境为python:3.9.6-slim-buster，完整错误回溯信息如下：

Traceback (most recent call last):
  File "/usr/local/bin/celery", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/site-packages/celery/__main__.py", line 15, in main
    sys.exit(_main())
  File "/usr/local/lib/python3.9/site-packages/celery/bin/celery.py", line 217, in main
    return celery(auto_envvar_prefix="CELERY")
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/celery/bin/base.py", line 134, in caller
    return f(ctx, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/celery/bin/worker.py", line 343, in worker
    worker = app.Worker(
  File "/usr/local/lib/python3.9/site-packages/celery/worker/worker.py", line 94, in __init__
    self.app.loader.init_worker()
  File "/usr/local/lib/python3.9/site-packages/celery/loaders/base.py", line 111, in init_worker
    self.import_default_modules()
  File "/usr/local/lib/python3.9/site-packages/celery/loaders/base.py", line 105, in import_default_modules
    raise response
  File "/usr/local/lib/python3.9/site-packages/celery/utils/dispatch/signal.py", line 276, in send
    response = receiver(signal=self, sender=sender, **named)
  File "/usr/local/lib/python3.9/site-packages/celery/fixups/django.py", line 82, in on_import_modules
    self.worker_fixup.validate_models()
  File "/usr/local/lib/python3.9/site-packages/celery/fixups/django.py", line 120, in validate_models
    self.django_setup()
  File "/usr/local/lib/python3.9/site-packages/celery/fixups/django.py", line 116, in django_setup
    django.setup()
  File "/usr/local/lib/python3.9/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/local/lib/python3.9/site-packages/django/apps/registry.py", line 124, in populate
    app_config.ready()
  File "/usr/local/lib/python3.9/site-packages/django/contrib/admin/apps.py", line 27, in ready
    self.module.autodiscover()
  File "/usr/local/lib/python3.9/site-packages/django/contrib/admin/__init__.py", line 50, in autodiscover
    autodiscover_modules("admin", register_to=site)
  File "/usr/local/lib/python3.9/site-packages/django/utils/module_loading.py", line 58, in autodiscover_modules
    import_module("%s.%s" % (app_config.name, module_to_search))
  File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/app/backend/apps/series/admin.py", line 22, in <module>
    from apps.modules.advert.tasks import (
  File "/app/backend/apps/modules/advert/tasks.py", line 7, in <module>
    from playwright.sync_api import sync_playwright
  File "/usr/local/lib/python3.9/site-packages/playwright/sync_api/__init__.py", line 25, in <module>
    import playwright.sync_api._generated
  File "/usr/local/lib/python3.9/site-packages/playwright/sync_api/_generated.py", line 25, in <module>
    from playwright._impl._accessibility import Accessibility as AccessibilityImpl
  File "/usr/local/lib/python3.9/site-packages/playwright/_impl/_accessibility.py", line 17, in <module>
    from playwright._impl._connection import Channel
  File "/usr/local/lib/python3.9/site-packages/playwright/_impl/_connection.py", line 35, in <module>
    from pyee import EventEmitter
  File "/usr/local/lib/python3.9/site-packages/pyee/__init__.py", line 120, in <module>
    from pyee.trio import TrioEventEmitter as _TrioEventEmitter  # noqa
  File "/usr/local/lib/python3.9/site-packages/pyee/trio.py", line 7, in <module>
    import trio
  File "/usr/local/lib/python3.9/site-packages/trio/__init__.py", line 18, in <module>
    from ._core import (
  File "/usr/local/lib/python3.9/site-packages/trio/_core/__init__.py", line 27, in <module>
    from ._run import (
  File "/usr/local/lib/python3.9/site-packages/trio/_core/_run.py", line 2452, in <module>
    from ._io_epoll import EpollIOManager as TheIOManager
  File "/usr/local/lib/python3.9/site-packages/trio/_core/_io_epoll.py", line 188, in <module>
    class EpollIOManager:
  File "/usr/local/lib/python3.9/site-packages/trio/_core/_io_epoll.py", line 189, in EpollIOManager
    _epoll = attr.ib(factory=select.epoll)
AttributeError: module 'select' has no attribute 'epoll'

恳请各位提供解决方案或思路。

内容的提问来源于stack exchange，提问作者Ekin Ertaç