为何多个Celery Worker进程会显示相同的Greenlet ID与线程信息？

我们在Linux环境下，基于python:3.9.5-slim-buster构建Docker容器，分别部署Flask服务和Celery Worker，启动命令如下：

# Flask服务启动命令
gunicorn --bind 0.0.0.0:5000 project:app --access-logfile -

# Celery Worker启动命令
celery -A project.celery worker --loglevel=info

两者共用同一代码文件，内容如下：

import os
import time
import threading

from flask import (
    Flask,
    jsonify,
)
from flask_sqlalchemy import SQLAlchemy
from greenlet import getcurrent
from celery import Celery

import logging
logging.basicConfig()
logging.getLogger('sqlalchemy.engine').setLevel(logging.INFO)

app = Flask(__name__)
app.config.from_object("project.config.Config")
db = SQLAlchemy(app)

celery = Celery(__name__)
celery.conf.broker_url = os.environ.get("CELERY_BROKER_URL", "redis://redis:6379")
celery.conf.result_backend = os.environ.get("CELERY_RESULT_BACKEND", "redis://redis:6379")


@app.route("/update", methods=['GET'])
def update():

    for i in range(4):
        celery_worker.delay()

    return jsonify(result="success")


@celery.task(name="celery_worker")
def celery_worker():
    time.sleep(1)
    print(getcurrent())
    time.sleep(1)
    print(getcurrent().__hash__())
    time.sleep(1)
    print(threading.local())
    time.sleep(1)
    print(threading.current_thread())
    time.sleep(1)
    print(os.getpid())

运行后日志显示，尽管任务运行在不同的ForkPoolWorker进程（可通过os.getpid()输出的不同PID确认），但所有任务输出的Greenlet对象、哈希值、线程本地对象及主线程信息完全一致：

celery_worker_1  | [2022-08-15 09:50:16,721: INFO/MainProcess] Task celery_worker[f38b203c-51e8-4242-9b3e-e466a208a395] received
celery_worker_1  | [2022-08-15 09:50:16,722: INFO/MainProcess] Task celery_worker[60afd98f-1139-4962-9aaf-4049261dae06] received
web_1            | 172.20.0.1 - - [15/Aug/2022:09:50:16 +0000] "GET /update HTTP/1.1" 200 21 "-" "curl/7.68.0"
celery_worker_1  | [2022-08-15 09:50:16,724: INFO/MainProcess] Task celery_worker[06feb5ee-4ece-4dfa-9d73-acf396be5161] received
celery_worker_1  | [2022-08-15 09:50:16,725: INFO/MainProcess] Task celery_worker[545068a2-8b28-454d-a8dc-47809adea845] received
celery_worker_1  | [2022-08-15 09:50:17,724: WARNING/ForkPoolWorker-1] <greenlet.greenlet object at 0x7f7102ff25c0 (otid=0x7f71038996c0) current active started main>
celery_worker_1  | [2022-08-15 09:50:17,724: WARNING/ForkPoolWorker-8] <greenlet.greenlet object at 0x7f7102ff25c0 (otid=0x7f71038996c0) current active started main>
celery_worker_1  | [2022-08-15 09:50:17,727: WARNING/ForkPoolWorker-3] <greenlet.greenlet object at 0x7f7102ff25c0 (otid=0x7f71038996c0) current active started main>
celery_worker_1  | [2022-08-15 09:50:17,727: WARNING/ForkPoolWorker-2] <greenlet.greenlet object at 0x7f7102ff25c0 (otid=0x7f71038996c0) current active started main>
celery_worker_1  | [2022-08-15 09:50:18,727: WARNING/ForkPoolWorker-1] 8757709894236
celery_worker_1  | [2022-08-15 09:50:18,727: WARNING/ForkPoolWorker-8] 8757709894236
celery_worker_1  | [2022-08-15 09:50:18,729: WARNING/ForkPoolWorker-3] 8757709894236
celery_worker_1  | [2022-08-15 09:50:18,730: WARNING/ForkPoolWorker-2] 8757709894236
celery_worker_1  | [2022-08-15 09:50:19,729: WARNING/ForkPoolWorker-1] <_thread._local object at 0x7f7101191c20>
celery_worker_1  | [2022-08-15 09:50:19,729: WARNING/ForkPoolWorker-8] <_thread._local object at 0x7f7101191c20>
celery_worker_1  | [2022-08-15 09:50:19,731: WARNING/ForkPoolWorker-3] <_thread._local object at 0x7f7101191c20>
celery_worker_1  | [2022-08-15 09:50:19,731: WARNING/ForkPoolWorker-2] <_thread._local object at 0x7f7101191c20>
celery_worker_1  | [2022-08-15 09:50:20,731: WARNING/ForkPoolWorker-1] <_MainThread(MainThread, started 140123376166720)>
celery_worker_1  | [2022-08-15 09:50:20,731: WARNING/ForkPoolWorker-8] <_MainThread(MainThread, started 140123376166720)>
celery_worker_1  | [2022-08-15 09:50:20,732: WARNING/ForkPoolWorker-2] <_MainThread(MainThread, started 140123376166720)>
celery_worker_1  | [2022-08-15 09:50:20,732: WARNING/ForkPoolWorker-3] <_MainThread(MainThread, started 140123376166720)>
celery_worker_1  | [2022-08-15 09:50:21,733: WARNING/ForkPoolWorker-8] 16
celery_worker_1  | [2022-08-15 09:50:21,733: WARNING/ForkPoolWorker-1] 9
celery_worker_1  | [2022-08-15 09:50:21,733: WARNING/ForkPoolWorker-2] 10
celery_worker_1  | [2022-08-15 09:50:21,734: WARNING/ForkPoolWorker-3] 11
celery_worker_1  | [2022-08-15 09:50:21,743: INFO/ForkPoolWorker-8] Task celery_worker[f38b203c-51e8-4242-9b3e-e466a208a395] succeeded in 5.019944864034187s: None
celery_worker_1  | [2022-08-15 09:50:21,743: INFO/ForkPoolWorker-2] Task celery_worker[06feb5ee-4ece-4dfa-9d73-acf396be5161] succeeded in 5.016862046031747s: None
celery_worker_1  | [2022-08-15 09:50:21,743: INFO/ForkPoolWorker-1] Task celery_worker[60afd98f-1139-4962-9aaf-4049261dae06] succeeded in 5.02064540295396s: None
celery_worker_1  | [2022-08-15 09:50:21,743: INFO/ForkPoolWorker-3] Task celery_worker[545068a2-8b28-454d-a8dc-47809adea845] succeeded in 5.017592608986888s: None

这一问题极具破坏性——项目中部分工具是按线程作用域设计的，它们会误判所有任务运行在同一线程中，进而引发数据共享、状态污染等严重问题。

内容的提问来源于stack exchange，提问作者dameshgarm