Django-mailbox访问.html属性触发重复数据库查询问题求助

使用django-mailbox时，访问Message对象的.html属性会触发重复的数据库查询。已通过select_related和prefetch_related预关联mailbox、message_used及attachments数据，但循环遍历查询集并访问message.html时，仍产生13条针对django_mailbox_messageattachment的重复查询。

相关代码 #

messagesDB = Message.objects.filter(mailbox=mailbox).select_related('mailbox', 'message_used').prefetch_related(Prefetch('attachments', queryset=MessageAttachment.objects.all(), to_attr='all_attachments'))
for message in messagesDB:
    print(message.html)

查询日志 #

SELECT ••• FROM "django_mailbox_messageattachment" WHERE "django_mailbox_messageattachment"."id" = 44 LIMIT 21
13 similar queries.


SELECT "django_mailbox_messageattachment"."id",
       "django_mailbox_messageattachment"."message_id",
       "django_mailbox_messageattachment"."headers",
       "django_mailbox_messageattachment"."document"
  FROM "django_mailbox_messageattachment"
 WHERE "django_mailbox_messageattachment"."id" = 44
 LIMIT 21 13 similar queries.

调用栈 #

C:\Users\user\.virtualenvs\dashboard-APxxHYiU\lib\site-packages\django\contrib\staticfiles\handlers.py in __call__(80)
  return self.application(environ, start_response)

C:\Users\user\.virtualenvs\dashboard-APxxHYiU\lib\site-packages\whitenoise\middleware.py in __call__(124)
  return self.get_response(request)

C:\Users\user\AppData\Local\Programs\Python\Python310\lib\contextlib.py in inner(79)
  return func(*args, **kwds)

C:\Users\user\.virtualenvs\dashboard-APxxHYiU\lib\site-packages\django\views\generic\base.py in view(104)
  return self.dispatch(request, *args, **kwargs)

C:\Users\user\.virtualenvs\dashboard-APxxHYiU\lib\site-packages\django\contrib\auth\mixins.py in dispatch(73)
  return super().dispatch(request, *args, **kwargs)

C:\Users\user\.virtualenvs\dashboard-APxxHYiU\lib\site-packages\django\contrib\auth\mixins.py in dispatch(109)
  return super().dispatch(request, *args, **kwargs)

C:\Users\user\.virtualenvs\dashboard-APxxHYiU\lib\site-packages\django\views\generic\base.py in dispatch(143)
  return handler(request, *args, **kwargs)

C:\Users\user\.virtualenvs\dashboard-APxxHYiU\lib\site-packages\django\views\generic\base.py in get(226)
  context = self.get_context_data(**kwargs)

C:\Users\user\Desktop\Задачник\ProjectsDashboard\dashboard\timetable\views.py in get_context_data(2109)
  messagesDB = self.get_message_telegram(self.mailbox)

C:\Users\user\Desktop\Задачник\ProjectsDashboard\dashboard\timetable\views.py in get_message_telegram(2058)
  print(message.html)

C:\Users\user\.virtualenvs\dashboard-APxxHYiU\lib\site-packages\django_mailbox\models.py in html(671)
  self.get_email_object(), 'text', 'html'

C:\Users\user\.virtualenvs\dashboard-APxxHYiU\lib\site-packages\django_mailbox\models.py in get_email_object(783)
  self._email_object = self._rehydrate(flat)

C:\Users\user\.virtualenvs\dashboard-APxxHYiU\lib\site-packages\django_mailbox\models.py in _rehydrate(683)
  self._rehydrate(part)

C:\Users\user\.virtualenvs\dashboard-APxxHYiU\lib\site-packages\django_mailbox\models.py in _rehydrate(683)
  self._rehydrate(part)

C:\Users\user\.virtualenvs\dashboard-APxxHYiU\lib\site-packages\django_mailbox\models.py in _rehydrate(687)
  attachment = MessageAttachment.objects.get(

C:\Users\user\.virtualenvs\dashboard-APxxHYiU\lib\site-packages\cacheops\query.py in get(327)
  return qs._no_monkey.get(qs, *args, **kwargs)

C:\Users\user\.virtualenvs\dashboard-APxxHYiU\lib\site-packages\cacheops\query.py in _fetch_all(250)
  return self._no_monkey._fetch_all(self)

问题原因 #

从调用栈可见，访问.html属性会触发get_email_object()方法，进而调用_rehydrate()。而_rehydrate()内部直接使用MessageAttachment.objects.get()查询附件，完全绕开了prefetch_related预加载的数据——预加载的附件存在实例的all_attachments或_prefetched_objects_cache中，但内部代码并未利用这些缓存，而是每次都发起新的数据库查询。

解决方案 #

方案1：Monkey Patch修改_rehydrate方法

通过动态修改django-mailbox的Message._rehydrate方法，让它优先从预加载的附件集合中获取数据，而非直接查数据库：

from django_mailbox.models import Message, MessageAttachment

original_rehydrate = Message._rehydrate

def patched_rehydrate(self, part):
    # 检查当前处理的part是否是附件
    attachment_id = part.get('attachment_id')
    if attachment_id:
        # 优先从预加载的all_attachments中查找
        if hasattr(self, 'all_attachments'):
            for att in self.all_attachments:
                if att.id == attachment_id:
                    return att
        # fallback到默认的attachments关联
        try:
            return self.attachments.get(id=attachment_id)
        except MessageAttachment.DoesNotExist:
            pass
    # 非附件场景，调用原方法
    return original_rehydrate(self, part)

# 替换原方法
Message._rehydrate = patched_rehydrate

方案2：全局缓存附件字典

在查询消息前，先把所有相关附件缓存到字典中，再修改_rehydrate方法优先读取该缓存：

from django_mailbox.models import Message, MessageAttachment

# 先查询所有需要的消息
messagesDB = Message.objects.filter(mailbox=mailbox).select_related('mailbox', 'message_used').prefetch_related('attachments')
# 获取所有关联的附件ID并缓存成字典
message_ids = messagesDB.values_list('id', flat=True)
attachment_cache = {
    att.id: att 
    for att in MessageAttachment.objects.filter(message_id__in=message_ids)
}

# 修改_rehydrate方法
original_rehydrate = Message._rehydrate

def patched_rehydrate(self, part):
    attachment_id = part.get('attachment_id')
    if attachment_id and attachment_id in attachment_cache:
        return attachment_cache[attachment_id]
    return original_rehydrate(self, part)

Message._rehydrate = patched_rehydrate

# 后续循环访问message.html就不会触发重复查询了
for message in messagesDB:
    print(message.html)

内容的提问来源于stack exchange，提问作者kekai