如何通过技能在Azure Cognitive Search中提取DocX/PDF元数据实现可筛选新鲜度

一、原生元数据提取说明 #

Azure Cognitive Search爬取Blob存储中的PDF和DocX文件时，会自动提取文件的原生元数据，这些元数据可直接通过/document/metadata_*路径访问，常用字段包括：

• metadata_author：文档作者
• metadata_creation_date：文档创建日期
• metadata_title：文档标题
• metadata_last_modified：文档最后修改日期

二、扩展技能集（Skillset）提取并映射元数据 #

你当前的技能集已使用ShaperSkill提取创建日期，只需扩展该技能的输入参数，即可同时提取其他所需元数据，修改后的技能配置如下：

{
  "@odata.context": ... ,
  "@odata.etag": ... ,
  "name": "freshness",
  "description": "Skillset to chunk documents, generate embeddings and extract metadata",
  "skills": [
    {
      ... // 保留原有分块、生成嵌入的技能配置
    },
    {
      "@odata.type": "#Microsoft.Skills.Util.ShaperSkill",
      "name": "#3",
      "description": "Extracts key metadata from the document",
      "context": "/document",
      "inputs": [
        {
          "name": "author",
          "source": "/document/metadata_author"
        },
        {
          "name": "creationDate",
          "source": "/document/metadata_creation_date"
        },
        {
          "name": "title",
          "source": "/document/metadata_title"
        },
        {
          "name": "lastModifiedDate",
          "source": "/document/metadata_last_modified"
        }
      ],
      "outputs": [
        {
          "name": "output",
          "targetName": "documentMetadata"
        }
      ]
    }
  ],
  "cognitiveServices": null,
  "knowledgeStore": null,
  "indexProjections": {
    "selectors": [
      {
        "targetIndexName": "freshness",
        "parentKeyFieldName": "parent_id",
        "sourceContext": "/document/pages/*",
        "mappings": [
          {
            "name": "author",
            "source": "/document/documentMetadata/author"
          },
          {
            "name": "creationDate",
            "source": "/document/documentMetadata/creationDate"
          },
          {
            "name": "title",
            "source": "/document/documentMetadata/title"
          },
          {
            "name": "lastModifiedDate",
            "source": "/document/documentMetadata/lastModifiedDate"
          }
        ]
      }
    ],
    "parameters": {
      "projectionMode": "skipIndexingParentDocuments"
    }
  },
  "encryptionKey": null
}

三、目标索引字段配置 #

确保你的freshness索引中已定义对应可筛选字段，字段配置示例（需匹配映射的字段名）：

{
  "name": "freshness",
  "fields": [
    ... // 保留原有字段（如嵌入向量、分块内容等）
    {
      "name": "author",
      "type": "Edm.String",
      "filterable": true,
      "searchable": true
    },
    {
      "name": "creationDate",
      "type": "Edm.DateTimeOffset",
      "filterable": true,
      "sortable": true
    },
    {
      "name": "title",
      "type": "Edm.String",
      "filterable": true,
      "searchable": true
    },
    {
      "name": "lastModifiedDate",
      "type": "Edm.DateTimeOffset",
      "filterable": true,
      "sortable": true
    }
  ]
}

四、新鲜度筛选使用示例 #

索引器运行完成后，即可通过$filter参数实现新鲜度或其他元数据筛选，例如：

• 筛选2024年1月1日之后创建的文档：
  $filter=creationDate ge 2024-01-01T00:00:00Z
• 筛选特定作者的文档：
  $filter=author eq 'John Doe'

内容的提问来源于stack exchange，提问作者Cheoncio