解决pybliometrics中Scopus400Error：超出服务级查询上限问题

问题背景 #

使用pybliometrics库对Scopus数据库进行跨年份检索时，采用复杂查询语句触发Scopus400Error，提示"Exceeds the maximum number allowed for the service level"。此前脚本曾成功运行数次，复现原作者查询逻辑仍报错。

问题代码 #

x = ScopusSearch(
    f'PUBYEAR IS {year} AND TITLE ( "ethanol upgrading" OR "Guerbet" OR "Lebedev" OR "ethanol to" ) AND NOT TITLE ( "reforming" OR "electrocatalysis" OR "photocatalysis" OR "homogeneous") AND TITLE-ABS-KEY ( "1-butanol" OR "1,3-butadiene" OR "n-butanol" OR "butadiene" OR "higher alcohols" OR "Guerbet" OR "Lebedev" OR "diethyl ether" OR "BTX" OR "buetenes" OR "butene" OR "Isobutene" ) AND SUBJAREA ( ceng OR chem OR ener OR engi OR envi OR mate ) AND NOT SUBJAREA ( medi OR nurs OR vete OR dent OR heal OR mult ) AND NOT SUBJAREA ( agri OR bioc OR immu OR neur OR phar ) AND NOT SUBJAREA ( arts OR busi OR deci OR econ OR psyc OR soci ) AND NOT SUBJAREA ( phys OR math ) AND DOCTYPE ( "AR" ) AND SRCTYPE ( j ) ',
    view="STANDARD")

错误信息 #

pybliometrics.scopus.exception.Scopus400Error: Exceeds the maximum number allowed for the service level

解决方案 #

1. 简化查询语句结构 #

将分散的NOT SUBJAREA条件合并为单个语句，直接减少查询长度和复杂度：

# 合并所有排除的学科领域到一个NOT SUBJAREA中
query = f'PUBYEAR IS {year} AND TITLE ( "ethanol upgrading" OR "Guerbet" OR "Lebedev" OR "ethanol to" ) AND NOT TITLE ( "reforming" OR "electrocatalysis" OR "photocatalysis" OR "homogeneous") AND TITLE-ABS-KEY ( "1-butanol" OR "1,3-butadiene" OR "n-butanol" OR "butadiene" OR "higher alcohols" OR "Guerbet" OR "Lebedev" OR "diethyl ether" OR "BTX" OR "buetenes" OR "butene" OR "Isobutene" ) AND SUBJAREA ( ceng OR chem OR ener OR engi OR envi OR mate ) AND NOT SUBJAREA ( medi OR nurs OR vete OR dent OR heal OR mult OR agri OR bioc OR immu OR neur OR phar OR arts OR busi OR deci OR econ OR psyc OR soci OR phys OR math ) AND DOCTYPE ( "AR" ) AND SRCTYPE ( j )'
x = ScopusSearch(query, view="STANDARD")

2. 拆分检索流程（分两次查询） #

先获取符合基础条件的文献ID列表，再基于ID过滤文本关键词，降低单次查询的复杂度：

# 第一步：检索符合学科、年份、文献类型的文献ID（仅获取ID，轻量查询）
first_query = f'PUBYEAR IS {year} AND SUBJAREA ( ceng OR chem OR ener OR engi OR envi OR mate ) AND NOT SUBJAREA ( medi OR nurs OR vete OR dent OR heal OR mult OR agri OR bioc OR immu OR neur OR phar OR arts OR busi OR deci OR econ OR psyc OR soci OR phys OR math ) AND DOCTYPE ( "AR" ) AND SRCTYPE ( j )'
first_search = ScopusSearch(first_query, view="ID_LIST")
eids = [item["eid"] for item in first_search.results]

# 第二步：按ID批量过滤标题和关键词条件（ID过多时分批次处理）
batch_size = 50
final_results = []
for i in range(0, len(eids), batch_size):
    batch_eids = eids[i:i+batch_size]
    eid_str = ",".join(batch_eids)
    second_query = f'EID ( {eid_str} ) AND TITLE ( "ethanol upgrading" OR "Guerbet" OR "Lebedev" OR "ethanol to" ) AND NOT TITLE ( "reforming" OR "electrocatalysis" OR "photocatalysis" OR "homogeneous") AND TITLE-ABS-KEY ( "1-butanol" OR "1,3-butadiene" OR "n-butanol" OR "butadiene" OR "higher alcohols" OR "Guerbet" OR "Lebedev" OR "diethyl ether" OR "BTX" OR "buetenes" OR "butene" OR "Isobutene" )'
    batch_search = ScopusSearch(second_query, view="STANDARD")
    final_results.extend(batch_search.results)

# final_results即为最终检索结果

3. 优化关键词逻辑 #

• 移除重复关键词：Guerbet和Lebedev同时出现在TITLE和TITLE-ABS-KEY中，可调整为仅在TITLE-ABS-KEY中保留（确保逻辑一致的前提下减少冗余）
• 简化宽泛关键词：比如"ethanol to"可替换为更具体的表述，缩小匹配范围，降低查询负载

4. 检查服务级别配额 #

• 机构用户：联系Scopus管理员确认当前服务级别是否有查询复杂度/结果数量限制，申请调整配额
• 个人免费用户：等待每日配额重置，或进一步简化查询以减少返回结果规模

内容的提问来源于stack exchange，提问作者Caio Henrique Pinheiro