如何在Python中实现Wikidata QID与实体的双向转换

前置依赖 #

先安装请求库用来调用Wikidata公开接口：
pip install requests

核心工具函数封装 #

双向转换通过Wikidata的实体查询接口实现，支持批量查询，减少请求频次，以下是可直接复用的代码：

import requests
import time

# 全局请求配置
HEADERS = {
    "User-Agent": "WikidataMapper/1.0 (fill in your real email here; for non-commercial use)"
}
WD_API = "https://www.wikidata.org/w/api.php"

def entity_name_to_qid(entity_names: list[str], lang: str = "en") -> dict[str, str | None]:
    """
    批量将实体标准名转换为Wikidata QID
    :param entity_names: 待转换的实体标准名列表
    :param lang: 实体名对应的语言，默认英文
    :return: 映射字典，key为输入的实体名，value为匹配到的QID，无匹配则返回None
    """
    res_map = {name: None for name in entity_names}
    batch_size = 50  # 单批次最多查询50个实体，避免请求过长
    for idx in range(0, len(entity_names), batch_size):
        batch = entity_names[idx:idx+batch_size]
        req_params = {
            "action": "wbgetentities",
            "sites": f"{lang}wiki",
            "titles": "|".join(batch),
            "props": "labels|aliases",
            "format": "json",
            "languages": lang
        }
        resp = requests.get(WD_API, params=req_params, headers=HEADERS)
        resp.raise_for_status()
        resp_data = resp.json()
        # 解析返回结果做精确匹配
        if "entities" in resp_data:
            for qid, ent_info in resp_data["entities"].items():
                if qid.startswith("-"):  # 跳过接口返回的不存在实体项
                    continue
                # 收集实体的所有匹配名：主标签+别名
                match_names = set()
                if "labels" in ent_info and lang in ent_info["labels"]:
                    match_names.add(ent_info["labels"][lang]["value"].lower())
                if "aliases" in ent_info and lang in ent_info["aliases"]:
                    for alias in ent_info["aliases"][lang]:
                        match_names.add(alias["value"].lower())
                # 和输入的实体名做匹配
                for input_name in batch:
                    if input_name.lower() in match_names:
                        res_map[input_name] = qid
        time.sleep(0.3)  # 加延迟避免触发频率限制
    return res_map

def qid_to_entity_name(qids: list[str], lang: str = "en") -> dict[str, str | None]:
    """
    批量将Wikidata QID转换为对应实体标准名
    :param qids: 待转换的QID列表
    :param lang: 返回实体名的语言，默认英文
    :return: 映射字典，key为输入的QID，value为匹配到的实体名，无匹配则返回None
    """
    res_map = {qid: None for qid in qids}
    batch_size = 50
    for idx in range(0, len(qids), batch_size):
        batch = qids[idx:idx+batch_size]
        req_params = {
            "action": "wbgetentities",
            "ids": "|".join(batch),
            "props": "labels",
            "format": "json",
            "languages": lang
        }
        resp = requests.get(WD_API, params=req_params, headers=HEADERS)
        resp.raise_for_status()
        resp_data = resp.json()
        if "entities" in resp_data:
            for qid in batch:
                if qid in resp_data["entities"]:
                    label_info = resp_data["entities"][qid].get("labels", {}).get(lang)
                    if label_info:
                        res_map[qid] = label_info["value"]
        time.sleep(0.3)
    return res_map

接口使用提示：请求时必须填写真实联系邮箱到User-Agent字段、单次请求批量提交条目、保持300ms以上的请求间隔，符合Wikidata公开接口使用规范，避免被临时封禁IP。

业务流程适配 #

针对抽取实体对、QID格式三元组的处理场景，按以下步骤调用即可：

1. 从anchored_et字段中提取所有实体标准名，批量转换为对应QID
2. 遍历自有三元组数据，筛选出主语属于目标QID集合的条目
3. 提取筛选后三元组中的所有宾语QID，批量转换为实体标准名，完成格式转换

对应可直接运行的示例代码：

# 输入示例：抽取得到的实体配对数据
anchored_et = [
    ["Anti-authoritarianism", "anti-authoritarian"],
    ["Political philosophy", "political"],
    ["Social philosophy", "social philosophy"],
    ["Hierarchy", "hierarchies"],
    ["Workers' self-management", "self-managed"],
    ["Self-governance", "self-governed"],
    ["cooperative", "cooperative"]
]

# 步骤1：实体名转QID
std_entity_names = [item[0] for item in anchored_et]
name_2_qid = entity_name_to_qid(std_entity_names)
target_qid_set = set(name_2_qid.values())

# 步骤2：筛选匹配的三元组（以下为模拟的自有三元组数据，格式为(主语QID, 关系, 宾语QID)）
raw_triples = [
    ("Q188442", "subclass of", "Q179805"),
    ("Q6116", "instance of", "Q179805"),
    ("Q42757", "opposed by", "Q188442")
]
matched_triples = [t for t in raw_triples if t[0] in target_qid_set]

# 步骤3：宾语QID转实体名
obj_qids = list({t[2] for t in matched_triples})
qid_2_name = qid_to_entity_name(obj_qids)
# 补充主语的名称映射，方便直接输出可读结果
sub_qids = list({t[0] for t in matched_triples})
qid_2_name.update(qid_to_entity_name(sub_qids))

# 组装最终可读结果
final_triples = []
for sub_qid, rel, obj_qid in matched_triples:
    final_triples.append(
        (
            qid_2_name.get(sub_qid, sub_qid),
            rel,
            qid_2_name.get(obj_qid, obj_qid)
        )
    )

常见问题处理 #

• 实体名匹配不到QID：优先检查实体名的拼写、语言参数是否正确，可额外增加模糊匹配逻辑，通过Wikidata的搜索接口补全召回
• 单实体名对应多个QID：结合文本所属领域加规则过滤，比如政治哲学类文本优先选择实体描述中包含领域关键词的条目，降低歧义
• 大批量数据处理：将查询得到的QID-实体名映射存入本地缓存（如json文件、SQLite数据库），重复查询时直接读取缓存，减少接口请求次数，提升处理效率

内容的提问来源于stack exchange，提问作者zahra mahani