如何跨不同MongoDB数据库合并同结构的stack_validation集合？

需要合并两个独立MongoDB数据库中结构完全一致的stack_validation集合，尝试过$unionWith、$lookup、$merge等操作符均无法满足需求，且受安全限制不能创建新的整合数据库。

集合详情 #

数据库A的stack_validation集合

[
  {
    "_id": "03fc2e61c0d6e351437ffd65820cd387",
    "clu_ids": [
      "637E5C41-43A8-41D3-9627-5D9A33028179"
    ],
    "clus_validation_ids": [
      {
        "$ref": "validation_clus",
        "$id": "637E5C41-43A8-41D3-9627-5D9A33028179"
      }
    ],
    "status": "New",
    "last_data_date": null,
    "date_updated": null,
    "flags": {
      "flag1": false,
      "flag2": false,
      "flag3": false,
      "flag4": false
    },
    "notes": [
      "This is a demo note for Stack"
    ]
  },
  {
    "_id": "03fc2e61c0d6e3asdsa51437ffd65820cd387",
    "clu_ids": [
      "637E5C41-43A8-41D3-9627-5D9A33028179"
    ],
    "clus_validation_ids": [
      {
        "$ref": "validation_clus",
        "$id": "637E5Ca41-43A8-41D3-9627-5D9A33028179"
      }
    ],
    "status": "New",
    "last_data_date": null,
    "date_updated": null,
    "flags": {
      "flag1": false,
      "flag2": false,
      "flag3": false,
      "flag4": false
    },
    "notes": [
      "This is a demo note for Stack"
    ]
  }
]

数据库B的stack_validation集合

[
  {
    "_id": "03fcasd2e61c0d6e351437ffd65820cd387",
    "clu_ids": [
      "637E5C41-43A8-41D3-das9627-5D9A33028179"
    ],
    "clus_validation_ids": [
      {
        "$ref": "validation_clus",
        "$id": "637E1235C41-43A8-41D3-9627-5D9A33028179"
      }
    ],
    "status": "New",
    "last_data_date": null,
    "date_updated": null,
    "flags": {
      "flag1": false,
      "flag2": false,
      "flag3": false,
      "flag4": false
    },
    "notes": [
      "This is a demo note for Stack"
    ]
  },
  {
    "_id": "03fc2e61c0d6e351437ffd65820cd387",
    "clu_ids": [
      "637E2121d41-43A8-41D3-9627-5D9A33028179"
    ],
    "clus_validation_ids": [
      {
        "$ref": "validation_clus",
        "$id": "637E52131-43A8-41D3-9627-5D9A33028179"
      }
    ],
    "status": "New",
    "last_data_date": null,
    "date_updated": null,
    "flags": {
      "flag1": false,
      "flag2": false,
      "flag3": false,
      "flag4": false
    },
    "notes": [
      "This is a demo note for Stack"
    ]
  }
]

预期合并结果 #

[
  {
    "_id": "03fcasd2e61c0d6e351437ffd65820cd387",
    "clu_ids": [
      "637E5C41-43A8-41D3-das9627-5D9A33028179"
    ],
    "clus_validation_ids": [
      {
        "$ref": "validation_clus",
        "$id": "637E1235C41-43A8-41D3-9627-5D9A33028179"
      }
    ],
    "status": "New",
    "last_data_date": null,
    "date_updated": null,
    "flags": {
      "flag1": false,
      "flag2": false,
      "flag3": false,
      "flag4": false
    },
    "notes": [
      "This is a demo note for Stack"
    ]
  },
  {
    "_id": "03fc2e61c0d6e351437ffd65820cd387",
    "clu_ids": [
      "637E2121d41-43A8-41D3-9627-5D9A33028179"
    ],
    "clus_validation_ids": [
      {
        "$ref": "validation_clus",
        "$id": "637E52131-43A8-41D3-9627-5D9A33028179"
      }
    ],
    "status": "New",
    "last_data_date": null,
    "date_updated": null,
    "flags": {
      "flag1": false,
      "flag2": false,
      "flag3": false,
      "flag4": false
    },
    "notes": [
      "This is a demo note for Stack"
    ]
  },
  {
    "_id": "03fc2e61c0d6e351437ffd65820cd387",
    "clu_ids": [
      "637E5C41-43A8-41D3-9627-5D9A33028179"
    ],
    "clus_validation_ids": [
      {
        "$ref": "validation_clus",
        "$id": "637E5C41-43A8-41D3-9627-5D9A33028179"
      }
    ],
    "status": "New",
    "last_data_date": null,
    "date_updated": null,
    "flags": {
      "flag1": false,
      "flag2": false,
      "flag3": false,
      "flag4": false
    },
    "notes": [
      "This is a demo note for Stack"
    ]
  },
  {
    "_id": "03fc2e61c0d6e3asdsa51437ffd65820cd387",
    "clu_ids": [
      "637E5C41-43A8-41D3-9627-5D9A33028179"
    ],
    "clus_validation_ids": [
      {
        "$ref": "validation_clus",
        "$id": "637E5Ca41-43A8-41D3-9627-5D9A33028179"
      }
    ],
    "status": "New",
    "last_data_date": null,
    "date_updated": null,
    "flags": {
      "flag1": false,
      "flag2": false,
      "flag3": false,
      "flag4": false
    },
    "notes": [
      "This is a demo note for Stack"
    ]
  }
]

解决方案：MongoDB Atlas数据联邦 #

Atlas数据联邦是MongoDB提供的一项服务，允许你在不复制、移动数据，也无需创建新整合数据库的前提下，跨多个MongoDB数据库、集群甚至外部存储系统创建统一的查询视图。它通过虚拟层将分散的集合合并，直接返回包含所有文档的结果（即使存在重复_id也会保留，完全匹配预期需求）。

核心操作步骤

• 在Atlas控制台中配置数据联邦，添加两个目标数据库作为数据源
• 创建虚拟集合，将其映射到两个数据库的stack_validation集合
• 直接查询该虚拟集合，即可获取合并后的完整数据集

内容的提问来源于stack exchange，提问作者Matan Lasry