基于物种与菌株名提取GenBank登录号的网页爬取问题（BeautifulSoup/Selenium）

你的代码仅定位到页面中零散包含"JCM 5058"的文本节点，未精准锁定组装信息的完整区块。以下是两种可行的改进方案，可提取目标的完整组装详情，甚至整理为结构化表格。

方案1：纯Selenium精准定位提取 #

直接定位搜索结果中的组装信息卡片，提取其中全部文本内容：

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

search_term = "Streptomyces anthocyanicus JCM 5058"
driver = webdriver.Chrome()

# 构造搜索URL
search_url = f"https://www.ncbi.nlm.nih.gov/assembly/?term={search_term.replace(' ', '+')}"
driver.get(search_url)

try:
    # 等待组装结果卡片加载完成，定位包含目标菌株信息的区块
    wait = WebDriverWait(driver, 10)
    assembly_card = wait.until(EC.presence_of_element_located((By.XPATH, "//div[contains(@class, 'summary') and .//span[contains(text(), 'JCM 5058')]]")))
    
    # 提取卡片内的完整文本
    full_info = assembly_card.text
    print("提取到的完整组装信息：")
    print(full_info)
    
finally:
    driver.quit()

方案2：Selenium+BeautifulSoup结构化提取（推荐） #

先获取页面源码，再用BeautifulSoup解析，方便提取结构化信息并整理为表格：

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from bs4 import BeautifulSoup
import pandas as pd

search_term = "Streptomyces anthocyanicus JCM 5058"
driver = webdriver.Chrome()

search_url = f"https://www.ncbi.nlm.nih.gov/assembly/?term={search_term.replace(' ', '+')}"
driver.get(search_url)

try:
    # 等待页面加载完成
    wait = WebDriverWait(driver, 10)
    wait.until(EC.presence_of_element_located((By.XPATH, "//div[contains(@class, 'summary')]")))
    
    # 获取页面源码并解析
    page_source = driver.page_source
    soup = BeautifulSoup(page_source, 'html.parser')
    
    # 定位目标组装卡片
    target_card = soup.find('div', class_='summary', string=lambda text: text and 'JCM 5058' in text)
    
    if target_card:
        # 整理信息为字典
        info_dict = {}
        # 提取组装版本号（ASM开头内容）
        asm_version = target_card.find_previous('div', class_='title').text.strip()
        info_dict['组装版本'] = asm_version
        
        # 遍历每行信息拆分键值对
        for line in target_card.text.split('\n'):
            line = line.strip()
            if ':' in line:
                key, value = line.split(':', 1)
                info_dict[key.strip()] = value.strip()
        
        # 转换为表格输出
        df = pd.DataFrame([info_dict])
        print("结构化表格输出：")
        print(df.to_string(index=False))
    else:
        print("未找到目标菌株的组装信息")
        
finally:
    driver.quit()

输出示例 #

组装版本          Organism                          Infraspecific name          Submitter                                      Date       Assembly level Genome representation Relation to type material GenBank assembly accession RefSeq assembly accession IDs
ASM1465115v1  Streptomyces anthocyanicus (high G+C Gram-positive bacteria)  Strain: JCM 5058  WFCC-MIRCEN World Data Centre for Microorganisms (WDCM)  2020/09/12  Scaffold               full                      assembly from type material  GCA_014651155.1 (latest)      GCF_014651155.1 (latest)      8121141 [UID] 22194358 [GenBank] 22446388 [RefSeq]

内容的提问来源于stack exchange，提问作者Umar