如何解决Python循环调用PubMed API时出现的KeyError问题？

问题原因 #

1. 单个字符串被错误迭代：主程序中row['Term']是单个搜索词字符串，传给get_pubmed_results_count后，函数里的for term in search_terms会把字符串拆成单个字符逐个处理，导致API请求无效搜索词，返回的JSON结构异常，触发KeyError。
2. 结果变量被重复覆盖：主循环中每次调用函数都会覆盖result_counts，最终仅保留最后一个搜索词的结果。
3. 结果处理逻辑冗余：在遍历result_counts.items()时重复创建并保存DataFrame，会多次覆盖CSV文件，完全没必要。

修正方案（推荐方式：批量传递搜索词列表） #

import requests
import time
import pandas as pd

def get_pubmed_results_count(search_terms, delay=1):
    base_url = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi"
    results = {}

    for term in search_terms:
        params = {
            "db": "pubmed",
            "term": term,
            "retmode": "json"
        }

        try:
            response = requests.get(base_url, params=params)
            response.raise_for_status()
            data = response.json()
            count = data['esearchresult']['count']
            results[term] = count
        except requests.exceptions.RequestException as e:
            print(f"获取'{term}'数据失败: {e}")
            results[term] = None
        except KeyError as e:
            print(f"解析'{term}'结果失败: 缺失键 {e}")
            results[term] = None

        time.sleep(delay)

    return results

if __name__ == "__main__":
    # 读取搜索词CSV
    df_searchterms = pd.read_csv('search1.csv')
    # 提取Term列转为列表
    search_terms_list = df_searchterms['Term'].tolist()
    
    # 批量获取结果
    result_counts = get_pubmed_results_count(search_terms_list)
    
    # 转换为DataFrame并保存
    df_results = pd.DataFrame(result_counts.items(), columns=['term', 'count'])
    print(df_results)
    df_results.to_csv('TestResults1.csv', index=False)

备选方案（逐行处理单个搜索词） #

如果需要逐行控制逻辑，可以修改函数处理单个搜索词：

import requests
import time
import pandas as pd

def get_pubmed_single_count(search_term, delay=1):
    base_url = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi"
    params = {
        "db": "pubmed",
        "term": search_term,
        "retmode": "json"
    }

    try:
        response = requests.get(base_url, params=params)
        response.raise_for_status()
        data = response.json()
        return data['esearchresult']['count']
    except requests.exceptions.RequestException as e:
        print(f"获取'{search_term}'数据失败: {e}")
        return None
    except KeyError as e:
        print(f"解析'{search_term}'结果失败: 缺失键 {e}")
        return None
    finally:
        time.sleep(delay)

if __name__ == "__main__":
    df_searchterms = pd.read_csv('search1.csv')
    results = []
    
    for index, row in df_searchterms.iterrows():
        term = row['Term']
        count = get_pubmed_single_count(term)
        results.append({'term': term, 'count': count})
    
    df_results = pd.DataFrame(results)
    print(df_results)
    df_results.to_csv('TestResults1.csv', index=False)

修正说明 #

• 确保传递给函数的是字符串列表，而非单个字符串，避免错误迭代字符。
• 添加KeyError捕获，避免API返回异常JSON导致程序崩溃。
• 将结果处理逻辑放在循环外，仅创建并保存一次DataFrame，避免重复覆盖文件。

内容的提问来源于stack exchange，提问作者Rae Van Sandt