Qdrant咨询：如何向现有集合增量添加图片避免覆盖数据

你遇到的问题根源是代码中使用了qclient.recreate_collection()方法——这个方法会直接删除指定名称的现有集合，再重新创建空集合，自然会丢失旧数据。Qdrant完全支持向现有集合增量添加任意数量的数据，只需调整集合初始化逻辑和ID生成方式即可。

修改方案： #

1. 检查集合是否存在，不存在再创建：用collection_exists判断集合状态，仅在集合不存在时执行创建操作
2. 避免ID冲突：新增图片时，先获取集合当前的已存记录数，从该值开始分配新记录的ID，防止覆盖旧数据
3. 修正向量格式：确保将模型输出的张量转为列表格式，符合Qdrant的向量存储要求

修改后的完整代码： #

class ImageEmbedding:
    def image_to_database(self):
        base_directory = "Images"
        all_image_urls = os.listdir(base_directory)
        sample_image_urls = list(map(lambda item: f"{base_directory}/{item}", all_image_urls))

        payloads = DataFrame.from_records({"image_url": sample_image_urls})
        payloads["model_id"] = 2

        target_width = 256

        def resize_image(image_url):
            pil_image = Image.open(image_url)
            image_aspect_ratio = pil_image.width / pil_image.height
            resized_pil_image = pil_image.resize([target_width, math.floor(target_width * image_aspect_ratio)])
            return resized_pil_image

        def convert_image_to_base64(pil_image):
            image_data = BytesIO()
            pil_image.save(image_data, format="JPEG")
            base64_string = base64.b64encode(image_data.getvalue()).decode("utf-8")
            return base64_string

        resized_images = list(map(lambda el: resize_image(el), sample_image_urls))
        base64_strings = list(map(lambda el: convert_image_to_base64(el), resized_images))
        payloads["base64"] = base64_strings

        processor = AutoImageProcessor.from_pretrained("microsoft/resnet-50")
        model = ResNetForImageClassification.from_pretrained("microsoft/resnet-50")

        images = list(map(lambda el: Image.open(el), payloads["image_url"]))
        inputs = processor(images, return_tensors="pt",)
        outputs = model(**inputs)
        embeddings = outputs.logits

        embedding_length = len(embeddings[0])

        load_dotenv()
        qclient = QdrantClient(
            url=os.getenv('QDRANT_DB_URL'),
            api_key=os.getenv('QDRANT_API_KEY'),
        )

        collection_name = "die_models_images"

        # 关键改动1：仅在集合不存在时创建
        if not qclient.collection_exists(collection_name=collection_name):
            qclient.create_collection(
                collection_name=collection_name,
                vectors_config=VectorParams(
                    size=embedding_length,
                    distance=Distance.COSINE
                )
            )

        # 关键改动2：获取当前集合已存记录数，作为新ID起始值
        try:
            collection_info = qclient.get_collection(collection_name=collection_name)
            start_id = collection_info.points_count
        except Exception:
            start_id = 0

        payload_dicts = payloads.to_dict(orient="records")

        # 关键改动3：分配不重复的ID，并将张量转为列表
        records = [
            models.Record(
                id=start_id + idx,
                payload=payload_dicts[idx],
                vector=embeddings[idx].tolist()
            )
            for idx, _ in enumerate(payload_dicts)
        ]

        qclient.upload_records(
            collection_name=collection_name,
            records=records
        )

关键改动说明： #

• 替换recreate_collection为collection_exists+create_collection，保留原有集合数据
• 通过get_collection获取已存记录数，确保新记录ID不会与旧数据重复
• 添加.tolist()将PyTorch张量转为普通列表，避免Qdrant存储时的序列化错误

内容的提问来源于stack exchange，提问作者Serhat Bilal