升级MariaDB至10.7.8后Celery无法标记任务为完成的问题求助

我最近把MariaDB版本升级到10.7.8之后，Celery在尝试标记任务为完成时开始持续抛出警告，折腾了好一阵子都没找到解决办法，特意来求助大家！

错误日志详情 #

WARNING - py.warnings - /usr/local/lib/python3.9/site-packages/celery/app/trace.py:660: RuntimeWarning: Exception raised outside body: StatementError('(sqlalchemy.exc.OperationalError) (MySQLdb._exceptions.OperationalError) (4091, "Unknown SEQUENCE: \'task_id_sequence\'")'):

Traceback (most recent call last):

File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1978, in _cursor_execute
self.dialect.do_execute(cursor, statement, parameters, context)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
cursor.execute(statement, parameters)
File "/usr/local/lib/python3.9/site-packages/MySQLdb/cursors.py", line 209, in execute
res = self._query(query)
File "/usr/local/lib/python3.9/site-packages/MySQLdb/cursors.py", line 315, in _query
db.query(q)
File "/usr/local/lib/python3.9/site-packages/MySQLdb/connections.py", line 239, in query
_mysql.connection.query(self, query)
MySQLdb._exceptions.OperationalError: (4091, "Unknown SEQUENCE: 'task_id_sequence'")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1800, in _execute_context
context = constructor(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 1026, in _init_compiled
self._process_executesingle_defaults()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 1928, in _process_executesingle_defaults
val = self.get_insert_default(c)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 1872, in get_insert_default
return self._exec_default(column, column.default, column.type)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 1733, in _exec_default
return self.fire_sequence(default, type_)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/dialects/mysql/base.py", line 1153, in fire_sequence
return self._execute_scalar(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 1320, in _execute_scalar
conn._cursor_execute(self.cursor, stmt, parameters, context=self)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1980, in _cursor_execute
self._handle_dbapi_exception(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 2124, in _handle_dbapi_exception
util.raise_(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
raise exception
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1978, in _cursor_execute
self.dialect.do_execute(cursor, statement, parameters, context)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
cursor.execute(statement, parameters)
File "/usr/local/lib/python3.9/site-packages/MySQLdb/cursors.py", line 209, in execute
res = self._query(query)
File "/usr/local/lib/python3.9/site-packages/MySQLdb/cursors.py", line 315, in _query
db.query(q)
File "/usr/local/lib/python3.9/site-packages/MySQLdb/connections.py", line 239, in query
_mysql.connection.query(self, query)
sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) (4091, "Unknown SEQUENCE: 'task_id_sequence'")

[SQL: select nextval(task_id_sequence)]

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

File "/usr/local/lib/python3.9/site-packages/celery/app/trace.py", line 518, in trace_task
task.backend.mark_as_done(
File "/usr/local/lib/python3.9/site-packages/celery/backends/base.py", line 162, in mark_as_done
self.store_result(task_id, result, state, request=request)
File "/usr/local/lib/python3.9/site-packages/celery/backends/base.py", line 528, in store_result
self._store_result(task_id, result, state, traceback,
File "/usr/local/lib/python3.9/site-packages/celery/backends/database/__init__.py", line 47, in _inner
return fun(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/celery/backends/database/__init__.py", line 123, in _store_result
session.flush()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 3444, in flush
self._flush(objects)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 3584, in _flush
transaction.rollback(_capture_exception=True)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
compat.raise_(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
raise exception
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 3544, in _flush
flush_context.execute()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute
rec.execute(self)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute
util.preloaded.orm_persistence.save_obj(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj
_emit_insert_statements(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements
result = connection._execute_20(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20
return meth(self, args_10style, kwargs_10style, execution_options)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/sql/elements.py", line 334, in _execute_on_connection
return connection._execute_clauseelement(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement
ret = self._execute_context(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1806, in _execute_context
self._handle_dbapi_exception(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 2124, in _handle_dbapi_exception
util.raise_(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
raise exception
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1800, in _execute_context
context = constructor(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 1026, in _init_compiled
self._process_executesingle_defaults()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 1928, in _process_executesingle_defaults
val = self.get_insert_default(c)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 1872, in get_insert_default
return self._exec_default(column, column.default, column.type)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 1733, in _exec_default
return self.fire_sequence(default, type_)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/dialects/mysql/base.py", line 1153, in fire_sequence
return self._execute_scalar(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 1320, in _execute_scalar
conn._cursor_execute(self.cursor, stmt, parameters, context=self)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1980, in _cursor_execute
self._handle_dbapi_exception(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 2124, in _handle_dbapi_exception
util.raise_(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
raise exception
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1978, in _cursor_execute
self.dialect.do_execute(cursor, statement, parameters, context)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
cursor.execute(statement, parameters)
File "/usr/local/lib/python3.9/site-packages/MySQLdb/cursors.py", line 209, in execute
res = self._query(query)
File "/usr/local/lib/python3.9/site-packages/MySQLdb/cursors.py", line 315, in _query
db.query(q)
File "/usr/local/lib/python3.9/site-packages/MySQLdb/connections.py", line 239, in query
_mysql.connection.query(self, query)
sqlalchemy.exc.StatementError: (sqlalchemy.exc.OperationalError) (MySQLdb._exceptions.OperationalError) (4091, "Unknown SEQUENCE: 'task_id_sequence'")

[SQL: INSERT INTO celery_taskmeta (id, task_id, status, result, date_done, traceback) VALUES (%s, %s, %s, %s, %s, %s)]
[parameters: [{'task_id': '2d5c94c0-d776-48b4-8875-460f7eaf4025', 'result': None, 'traceback': None}]]

当前使用的软件版本 #

• celery: 5.2.7
• sqlalchemy: 1.4.46
• mysqlclient: 1.4.6
• pymysql: 1.0.2

我搜了一圈都没找到相关的解决办法，猜测应该是需要对数据库表做一些操作来消除这个错误，求各位大佬指点！

可能的解决方案（社区常见思路） #

从错误信息来看，核心问题是Celery的数据库后端尝试调用task_id_sequence序列时找不到它，这大概率和MariaDB 10.7+对序列的处理变化有关，或是之前的表结构没适配新版本。可以试试下面几个步骤：

1. 手动创建缺失的序列
   直接在MariaDB中执行SQL语句创建对应的序列：

CREATE SEQUENCE task_id_sequence START WITH 1 INCREMENT BY 1;

创建完成后重启Celery服务，看看警告是否消失。

2. 将主键改为自增字段（更符合MySQL/MariaDB习惯）
   MariaDB和MySQL通常更常用自增主键而非序列，你可以修改celery_taskmeta表的主键类型：

ALTER TABLE celery_taskmeta MODIFY COLUMN id INT AUTO_INCREMENT;

这样后续插入数据时就会自动使用自增ID，不再依赖序列。

3. 确保SQLAlchemy使用MariaDB方言
   检查你的数据库连接字符串，确保指定了MariaDB的方言而不是MySQL，比如：

# 示例连接字符串
SQLALCHEMY_DATABASE_URI = "mariadb+mariadbconnector://user:password@host/dbname"

这样SQLAlchemy会针对MariaDB的特性进行适配，避免序列相关的兼容性问题。

4. 升级Celery到新版本
   Celery 5.2.7可能对MariaDB 10.7的支持不够完善，尝试升级到较新的版本（比如5.3.x系列），看看官方是否修复了这个兼容性问题。

备注：内容来源于stack exchange，提问作者Sublimer