请求解决Django_mailbox访问message.html时重复数据库查询问题

问题背景 #

访问Message对象的.html属性时，会触发大量重复的数据库查询，即便已经通过select_related和prefetch_related预加载了关联数据。

原代码 #

messagesDB = Message.objects.filter(mailbox=mailbox).select_related('mailbox', 'message_used').prefetch_related(Prefetch('attachments', queryset=MessageAttachment.objects.all(), to_attr='all_attachments'))
for message in messagesDB:
    print(message.html)

重复查询日志 #

SELECT ••• FROM "django_mailbox_messageattachment" WHERE "django_mailbox_messageattachment"."id" = 44 LIMIT 21
 13 similar queries.


SELECT "django_mailbox_messageattachment"."id",
       "django_mailbox_messageattachment"."message_id",
       "django_mailbox_messageattachment"."headers",
       "django_mailbox_messageattachment"."document"
  FROM "django_mailbox_messageattachment"
 WHERE "django_mailbox_messageattachment"."id" = 44
 LIMIT 21 13 similar queries.

调用栈 #

C:\Users\user\.virtualenvs\dashboard-APxxHYiU\lib\site-packages\django\contrib\staticfiles\handlers.py in __call__(80)
  return self.application(environ, start_response)

C:\Users\user\.virtualenvs\dashboard-APxxHYiU\lib\site-packages\whitenoise\middleware.py in __call__(124)
  return self.get_response(request)

C:\Users\user\AppData\Local\Programs\Python\Python310\lib\contextlib.py in inner(79)
  return func(*args, **kwds)

C:\Users\user\.virtualenvs\dashboard-APxxHYiU\lib\site-packages\django\views\generic\base.py in view(104)
  return self.dispatch(request, *args, **kwargs)

C:\Users\user\.virtualenvs\dashboard-APxxHYiU\lib\site-packages\django\contrib\auth\mixins.py in dispatch(73)
  return super().dispatch(request, *args, **kwargs)

C:\Users\user\.virtualenvs\dashboard-APxxHYiU\lib\site-packages\django\contrib\auth\mixins.py in dispatch(109)
  return super().dispatch(request, *args, **kwargs)

C:\Users\user\.virtualenvs\dashboard-APxxHYiU\lib\site-packages\django\views\generic\base.py in dispatch(143)
  return handler(request, *args, **kwargs)

C:\Users\user\.virtualenvs\dashboard-APxxHYiU\lib\site-packages\django\views\generic\base.py in get(226)
  context = self.get_context_data(**kwargs)

C:\Users\user\Desktop\Задачник\ProjectsDashboard\dashboard\timetable\views.py in get_context_data(2109)
  messagesDB = self.get_message_telegram(self.mailbox)

C:\Users\user\Desktop\Задачник\ProjectsDashboard\dashboard\timetable\views.py in get_message_telegram(2058)
  print(message.html)

C:\Users\user\.virtualenvs\dashboard-APxxHYiU\lib\site-packages\django_mailbox\models.py in html(671)
  self.get_email_object(), 'text', 'html'

C:\Users\user\.virtualenvs\dashboard-APxxHYiU\lib\site-packages\django_mailbox\models.py in get_email_object(783)
  self._email_object = self._rehydrate(flat)

C:\Users\user\.virtualenvs\dashboard-APxxHYiU\lib\site-packages\django_mailbox\models.py in _rehydrate(683)
  self._rehydrate(part)

C:\Users\user\.virtualenvs\dashboard-APxxHYiU\lib\site-packages\django_mailbox\models.py in _rehydrate(683)
  self._rehydrate(part)

C:\Users\user\.virtualenvs\dashboard-APxxHYiU\lib\site-packages\django_mailbox\models.py in _rehydrate(687)
  attachment = MessageAttachment.objects.get(

C:\Users\user\.virtualenvs\dashboard-APxxHYiU\lib\site-packages\cacheops\query.py in get(327)
  return qs._no_monkey.get(qs, *args, **kwargs)

C:\Users\user\.virtualenvs\dashboard-APxxHYiU\lib\site-packages\cacheops\query.py in _fetch_all(250)
  return self._no_monkey._fetch_all(self)

解决方案 #

从调用栈可知，问题根源在于django_mailbox的_rehydrate方法重建邮件对象时，直接调用MessageAttachment.objects.get()查询附件，完全未使用预加载的all_attachments属性，导致重复查询。以下是可行的解决方法：

方法1：猴子补丁修改_rehydrate逻辑 #

通过补丁让_rehydrate优先使用预加载的附件，避免重复查询：

from django_mailbox.models import Message

original_rehydrate = Message._rehydrate

def patched_rehydrate(self, part):
    if hasattr(self, 'all_attachments') and part.get('attachment_id'):
        # 从预加载的all_attachments中匹配附件
        for attachment in self.all_attachments:
            if attachment.id == part['attachment_id']:
                part['payload'] = attachment.document.read()
                return part
    # 原逻辑兜底
    return original_rehydrate(self, part)

Message._rehydrate = patched_rehydrate

将这段代码放在项目初始化位置（如settings.py末尾、apps.py的ready方法），确保启动时生效。

方法2：预加载后缓存附件内容 #

提前读取附件内容并缓存到Message对象，再通过补丁读取缓存：

# 查询时预加载附件
messagesDB = Message.objects.filter(mailbox=mailbox).select_related('mailbox', 'message_used').prefetch_related(Prefetch('attachments', queryset=MessageAttachment.objects.all(), to_attr='all_attachments'))

# 提前缓存附件内容
for message in messagesDB:
    message._cached_attachments = {
        att.id: att.document.read() for att in message.all_attachments
    }
    print(message.html)

搭配对应的猴子补丁：

from django_mailbox.models import Message

original_rehydrate = Message._rehydrate

def patched_rehydrate(self, part):
    if hasattr(self, '_cached_attachments') and part.get('attachment_id'):
        part['payload'] = self._cached_attachments.get(part['attachment_id'], b'')
        return part
    return original_rehydrate(self, part)

Message._rehydrate = patched_rehydrate

方法3：修改django_mailbox源码（不推荐） #

直接修改django_mailbox/models.py中的_rehydrate方法，将attachment = MessageAttachment.objects.get(id=part['attachment_id'])替换为从预加载关联属性中查找。但该方法会在包更新后失效，仅适合临时应急或fork维护场景。

内容的提问来源于stack exchange，提问作者kekai