如何使用RSelenium点击网页按钮并下载CSV文件？

问题描述 #

我需要从UKRI GTR网站抓取气候变化相关项目数据，目标是点击页面上的CSV按钮下载文件，对应的链接和初始代码如下：

目标链接与依赖加载 #

library(netstat)
library(RSelenium)

# 原代码URL未加引号，属于语法错误，已修正
url <- "https://gtr.ukri.org/search/project?term=%22climate+change%22+OR+%22climate+crisis%22&fetchSize=25&selectedSortableField=&selectedSortOrder=&fields=pro.gr%2Cpro.t%2Cpro.a%2Cpro.orcidId%2Cper.fn%2Cper.on%2Cper.sn%2Cper.fnsn%2Cper.orcidId%2Cper.org.n%2Cper.pro.t%2Cper.pro.abs%2Cpub.t%2Cpub.a%2Cpub.orcidId%2Corg.n%2Corg.orcidId%2Cacp.t%2Cacp.d%2Cacp.i%2Cacp.oid%2Ckf.d%2Ckf.oid%2Cis.t%2Cis.d%2Cis.oid%2Ccol.i%2Ccol.d%2Ccol.c%2Ccol.dept%2Ccol.org%2Ccol.pc%2Ccol.pic%2Ccol.oid%2Cip.t%2Cip.d%2Cip.i%2Cip.oid%2Cpol.i%2Cpol.gt%2Cpol.in%2Cpol.oid%2Cprod.t%2Cprod.d%2Cprod.i%2Cprod.oid%2Crtp.t%2Crtp.d%2Crtp.i%2Crtp.oid%2Crdm.t%2Crdm.d%2Crdm.i%2Crdm.oid%2Cstp.t%2Cstp.d%2Cstp.i%2Cstp.oid%2Cso.t%2Cso.d%2Cso.cn%2Cso.i%2Cso.oid%2Cff.t%2Cff.d%2Cff.c%2Cff.org%2Cff.dept%2Cff.oid%2Cdis.t%2Cdis.d%2Cdis.i%2Cdis.oid%2Ccpro.rtpc%2Ccpro.rcpgm%2Ccpro.hlt&type=#/csvConfirm"

当前报错的代码 #

rD <- rsDriver(port= free_port(), browser = "chrome", chromever = "106.0.5249.21", check = TRUE, verbose = TRUE)
remote_driver <- rD[["client"]] 
remDr <- rD$client
remDr$navigate(url)

# 原选择器无法定位到正确的CSV下载按钮
webElem <- remDr$findElement(using = "css", "content gtr-body d-flex flex-column ng-scope")
webElem$clickElement()

解决方案 #

核心问题分析 #

1. URL语法错误：原代码中URL未加双引号，直接触发R语法错误。
2. 元素定位错误：原选择器指向页面容器而非实际的CSV下载按钮。
3. 页面加载未等待：GTR网站基于AngularJS渲染，直接查找元素会因页面未加载完成而失败。

修正后的完整代码 #

library(netstat)
library(RSelenium)

# 1. 正确赋值URL（已添加引号）
url <- "https://gtr.ukri.org/search/project?term=%22climate+change%22+OR+%22climate+crisis%22&fetchSize=25&selectedSortableField=&selectedSortOrder=&fields=pro.gr%2Cpro.t%2Cpro.a%2Cpro.orcidId%2Cper.fn%2Cper.on%2Cper.sn%2Cper.fnsn%2Cper.orcidId%2Cper.org.n%2Cper.pro.t%2Cper.pro.abs%2Cpub.t%2Cpub.a%2Cpub.orcidId%2Corg.n%2Corg.orcidId%2Cacp.t%2Cacp.d%2Cacp.i%2Cacp.oid%2Ckf.d%2Ckf.oid%2Cis.t%2Cis.d%2Cis.oid%2Ccol.i%2Ccol.d%2Ccol.c%2Ccol.dept%2Ccol.org%2Ccol.pc%2Ccol.pic%2Ccol.oid%2Cip.t%2Cip.d%2Cip.i%2Cip.oid%2Cpol.i%2Cpol.gt%2Cpol.in%2Cpol.oid%2Cprod.t%2Cprod.d%2Cprod.i%2Cprod.oid%2Crtp.t%2Crtp.d%2Crtp.i%2Crtp.oid%2Crdm.t%2Crdm.d%2Crdm.i%2Crdm.oid%2Cstp.t%2Cstp.d%2Cstp.i%2Cstp.oid%2Cso.t%2Cso.d%2Cso.cn%2Cso.i%2Cso.oid%2Cff.t%2Cff.d%2Cff.c%2Cff.org%2Cff.dept%2Cff.oid%2Cdis.t%2Cdis.d%2Cdis.i%2Cdis.oid%2Ccpro.rtpc%2Ccpro.rcpgm%2Ccpro.hlt&type=#/csvConfirm"

# 2. 配置Chrome自动下载（可选，避免弹窗）
chrome_options <- list(
  chromeOptions = list(
    prefs = list(
      "download.default_directory" = "C:/your/download/path", # 替换为你的本地下载目录
      "download.prompt_for_download" = FALSE,
      "download.directory_upgrade" = TRUE
    )
  )
)

# 3. 启动Selenium驱动
rD <- rsDriver(
  port = free_port(),
  browser = "chrome",
  chromever = "106.0.5249.21",
  extraCapabilities = chrome_options,
  check = TRUE,
  verbose = TRUE
)
remDr <- rD$client

# 4. 导航到页面并等待渲染完成
remDr$navigate(url)
Sys.sleep(3) # 根据网络速度调整等待时间

# 5. 定位并点击CSV下载按钮
# 方式1：通过XPath匹配按钮文本
csv_button <- remDr$findElement(using = "xpath", "//button[contains(text(), 'Download CSV')]")
# 方式2：通过CSS选择器定位（若按钮有固定类）
# csv_button <- remDr$findElement(using = "css", "button.gtr-btn-primary")

csv_button$clickElement()

# 6. 等待下载完成后关闭驱动
Sys.sleep(5)
remDr$close()
rD$server$stop()

关键说明 #

• 元素定位：通过XPath匹配按钮文本是最可靠的方式，避免因页面类名变更导致定位失败。
• 页面等待：添加Sys.sleep()确保AngularJS页面完全渲染，也可使用remDr$waitForElement()实现更精准的显式等待。
• 自动下载配置：指定下载目录后，文件会直接保存到目标路径，无需手动确认弹窗。

内容的提问来源于stack exchange，提问作者jmutua