Django集成MongoEngine时出现弱引用对象不存在错误

我在Django应用中使用MongoEngine，Mongo中的每个文档都包含metadata字典，其中frame_url字段缺少base_url。我在Django序列化器中为其添加base_url，但偶尔会触发错误。

序列化器代码 #

def to_representation(self, instance):
    # when we search by image result has similarity and it`s milvus similarity and when search by text,
    # we show it`s reference similarity
    # obj.similarity: we had image and have similarity from milvus.
    # obj.metadata.get('similarity') :we don`t have similarity from milvus use matched item similarity.
    data = super(ResultSerializer, self).to_representation(instance)
    if instance.similarity is not None:
        data['similarity'] = decimal.Decimal(distance_to_score(instance.similarity)).quantize(
            decimal.Decimal('0.000'), rounding=decimal.ROUND_DOWN)
    else:
        data['similarity'] = instance.metadata.get('similarity')

    if frame_url := instance.metadata.get('frame_url'):
        data.get('metadata')['frame_url'] = SHAHIN_SETTING['base_url'] + frame_url
    return data

错误信息 #

Jan 23 16:17:10.527 | develop-shahin-develop-86df584cf6-wtrcm | paginated_data['results'] = ResultSerializer(el_result, many=True).data
Jan 23 16:17:10.527 | develop-shahin-develop-86df584cf6-wtrcm | File "/usr/local/lib/python3.8/site-packages/rest_framework/serializers.py", line 745, in data
Jan 23 16:17:10.527 | develop-shahin-develop-86df584cf6-wtrcm | ret = super().data
Jan 23 16:17:10.527 | develop-shahin-develop-86df584cf6-wtrcm | File "/usr/local/lib/python3.8/site-packages/rest_framework/serializers.py", line 246, in data
Jan 23 16:17:10.527 | develop-shahin-develop-86df584cf6-wtrcm | self._data = self.to_representation(self.instance)
Jan 23 16:17:10.527 | develop-shahin-develop-86df584cf6-wtrcm | File "/usr/local/lib/python3.8/site-packages/rest_framework/serializers.py", line 663, in to_representation
Jan 23 16:17:10.527 | develop-shahin-develop-86df584cf6-wtrcm | return [
Jan 23 16:17:10.527 | develop-shahin-develop-86df584cf6-wtrcm | File "/usr/local/lib/python3.8/site-packages/rest_framework/serializers.py", line 664, in <listcomp>
Jan 23 16:17:10.527 | develop-shahin-develop-86df584cf6-wtrcm | self.child.to_representation(item) for item in iterable
Jan 23 16:17:10.527 | develop-shahin-develop-86df584cf6-wtrcm | File "/app/apps/search/serializers/v2.py", line 67, in to_representation
Jan 23 16:17:10.527 | develop-shahin-develop-86df584cf6-wtrcm | data.get('metadata')['frame_url'] = SHAHIN_SETTING['base_url'] + frame_url
Jan 23 16:17:10.527 | develop-shahin-develop-86df584cf6-wtrcm | File "/usr/local/lib/python3.8/site-packages/mongoengine/base/datastructures.py", line 35, in wrapper
Jan 23 16:17:10.528 | develop-shahin-develop-86df584cf6-wtrcm | self._mark_as_changed(key)
Jan 23 16:17:10.528 | develop-shahin-develop-86df584cf6-wtrcm | File "/usr/local/lib/python3.8/site-packages/mongoengine/base/datastructures.py", line 98, in _mark_as_changed
Jan 23 16:17:10.528 | develop-shahin-develop-86df584cf6-wtrcm | if hasattr(self._instance, "_mark_as_changed"):
Jan 23 16:17:10.528 | develop-shahin-develop-86df584cf6-wtrcm | ReferenceError: weakly-referenced object no longer exists

解决方案 #

问题根源是data.get('metadata')返回的是MongoEngine内置的绑定型数据结构（比如EmbeddedDocumentDict），它持有原MongoDB文档实例的弱引用。当原实例被Python垃圾回收机制回收后，修改这个结构就会触发弱引用不存在的错误。

修复方案1：将metadata转为普通Python字典后再修改 #

直接复制metadata为普通字典，切断和原实例的绑定：

if frame_url := instance.metadata.get('frame_url'):
    # 把MongoEngine的绑定结构转为普通字典
    metadata = dict(data.get('metadata')) if data.get('metadata') else {}
    metadata['frame_url'] = SHAHIN_SETTING['base_url'] + frame_url
    data['metadata'] = metadata

修复方案2：在序列化器中指定metadata为普通字典字段 #

如果是ModelSerializer，可以直接在字段定义里用DictField，让父类序列化时直接返回普通字典：

class ResultSerializer(serializers.ModelSerializer):
    metadata = serializers.DictField()
    # 其他字段定义...

    def to_representation(self, instance):
        # 原逻辑不变，此时data['metadata']已经是普通字典
        data = super().to_representation(instance)
        # ... 其他处理逻辑
        if frame_url := instance.metadata.get('frame_url'):
            data['metadata']['frame_url'] = SHAHIN_SETTING['base_url'] + frame_url
        return data

修复方案3：直接从实例复制metadata并修改 #

跳过序列化后的data['metadata']，直接从instance.metadata复制出普通字典后修改：

if frame_url := instance.metadata.get('frame_url'):
    updated_metadata = dict(instance.metadata)
    updated_metadata['frame_url'] = SHAHIN_SETTING['base_url'] + frame_url
    data['metadata'] = updated_metadata

内容的提问来源于stack exchange，提问作者Mahnaz Divargar