Django/PostgreSQL随机触发Operational Error：连接关闭问题求助

我们碰到一个无规律发作的棘手问题：Django服务偶尔会抛出OperationalError: the connection is closed错误，错误随机出现后又自行消失。

运行环境：Django 4.2 / Python 3.11 - PgBouncer - PostgreSQL 14（部署在AWS EC2 + RDS）。此前无此类问题，升级到Django 4版本后才偶尔出现，业务仅执行常规Django create/save操作。

是否有用户遇到过类似情况，或能提供问题原因的线索？

错误栈 #

Error message

OperationalError: the connection is closed
  File "django/db/backends/base/base.py", line 308, in _cursor
    return self._prepare_cursor(self.create_cursor(name))
  File "django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "django/db/backends/postgresql/base.py", line 331, in create_cursor
    cursor = self.connection.cursor()
  File "psycopg/connection.py", line 840, in cursor
    self._check_connection_ok()
  File "psycopg/connection.py", line 479, in _check_connection_ok
    raise e.OperationalError("the connection is closed")

OperationalError: the connection is closed

Django数据库配置 #

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        ...
        "DISABLE_SERVER_SIDE_CURSORS": True,
        "CONN_MAX_AGE": None,
        "CONN_HEALTH_CHECKS": True
    },
 }

Gunicorn配置 #

[gunicorn]
workers = multiprocessing.cpu_count() * 2 + 1
keepalive = 120
timeout = 120
graceful_timeout = 120
worker_connections = 1000
worker_class = 'sync'

PgBouncer配置 #

[pgbouncer]
pool_mode = transaction

default_pool_size = 100
min_pool_size = 20
reserve_pool_size = 30
reserve_pool_timeout = 1

max_client_conn = 10000
server_reset_query =
server_check_delay = 0
server_tls_sslmode = verify-full
server_tls_ca_file = /etc/pgbouncer/xxxxxx

log_connections = 1
log_disconnections = 1
log_pooler_errors = 1

日志信息 #

PgBouncer日志（仅关键内容）

Apr 09 14:54:13 pgbouncer[719032]: stats: 20 xacts/s, 20 queries/s, in 25996 B/s, out 46656 B/s, xact 3349 us, query 3343 us, wait 24409 us
 Apr 09 14:55:13 pgbouncer[719032]: stats: 40 xacts/s, 40 queries/s, in 61480 B/s, out 229186 B/s, xact 3216 us, query 3162 us, wait 46999 us
 Apr 09 14:56:13 pgbouncer[719032]: stats: 22 xacts/s, 22 queries/s, in 34561 B/s, out 77043 B/s, xact 3076 us, query 3060 us, wait 22044 us
 Apr 09 14:57:13 pgbouncer[719032]: stats: 26 xacts/s, 26 queries/s, in 42978 B/s, out 185932 B/s, xact 3422 us, query 3407 us, wait 29930 us
 Apr 09 14:57:26 pgbouncer[719032]: C-0x560e05b89020: pgbouncer/netdata@unix(1806513):6432 closing because: client close request (age=600s)
 Apr 09 14:57:26 pgbouncer[719032]: C-0x560e05b89020: pgbouncer/netdata@unix(1806513):6432 login attempt: db=pgbouncer user=netdata tls=no
 Apr 09 14:58:13 pgbouncer[719032]: stats: 21 xacts/s, 21 queries/s, in 37727 B/s, out 211338 B/s, xact 3593 us, query 3523 us, wait 27558 us
 Apr 09 14:58:26 pgbouncer[719032]: S-0x560e05b97210: shoot_dev/shoot_dev@172.31.29.188:5432 closing because: server lifetime over (age=3600s)

PostgreSQL日志

仅记录checkpoint开始/完成信息，无其他异常。

内容的提问来源于stack exchange，提问作者Jens Lundstrom