如何用R语言RSelenium包获取含tablist类div标签的页面谷歌地理坐标
问题
尝试用R的RSelenium包获取目标页面的地理坐标(目标值:20º27'36.1"S 54º38'03.1"W),三次尝试均失败,代码如下:
library(rvest) library(RSelenium) library(httpuv) port <- httpuv::randomPort() rD <- rsDriver(browser = c("firefox"), verbose=TRUE, check = FALSE, port = port) driver <- rD[["client"]] urll <- "https://www.zapimoveis.com.br/lancamento/venda-apartamento-2-quartos-bairro-seminario-campo-grande-ms-46m2-id-2600496487/" driver$navigate(urll) politicas <- driver$findElement(using = "css", value = "button.cookie-notifier__cta") politicas$clickElement() botaomapa <- driver$findElement(using = "xpath", "/html/body/main/div[1]/section/section/section[1]/button[2]") botaomapa$clickElement() #Attempt 1: using xpath from coordinates coord <- driver$findElement(using="xpath", "/html/body/div/div/div/div[4]/div/div/div/div/div[1]/div")#errorrrrr #Attempt 2: by botaomapa object coord <- botaomapa$findElement(using="xpath", "/html/body/div/div/div/div[4]/div/div/div/div/div[1]/div")#errorrrr #Attempt 3: by rvest package readmap <- read_html(urll) auxiliar <- readmap %>% html_elements("section") auxiliar2 <- auxiliar%>%html_elements("#listing-map") c1 <- readmap%>%html_nodes(xpath="/html/body/main/div[1]/section/section/section[2]/div/div[3]/article/iframe")#nothing c2 <- auxiliar2%>%html_nodes(xpath="/html/body/main/div[1]/section/section/section[2]/div/div[3]/article/iframe")#nothing c3 <- auxiliar2%>%html_nodes(xpath="/html/body/div/div/div/div[4]/div/div/div/div/div[1]/div")#nothing
问题原因
- 绝对XPATH不稳定:使用
/html/body/...这种绝对路径,页面结构稍有变动就会失效,且无法定位到iframe内的元素 - 未处理iframe:点击地图按钮后,地图内容加载在iframe中,直接在主页面上下文无法定位iframe内的元素
- rvest局限性:rvest只能抓取初始静态HTML,无法获取动态加载的iframe内容,也无法模拟点击交互
解决方案
步骤如下:
- 处理cookie弹窗后,点击地图按钮打开模态框
- 切换到地图所在的iframe上下文
- 定位坐标显示元素并提取文本
- 完成后关闭驱动
修正后的代码:
library(RSelenium) library(httpuv) # 初始化驱动 port <- httpuv::randomPort() rD <- rsDriver(browser = "firefox", verbose = TRUE, check = FALSE, port = port) driver <- rD[["client"]] # 访问目标页面 urll <- "https://www.zapimoveis.com.br/lancamento/venda-apartamento-2-quartos-bairro-seminario-campo-grande-ms-46m2-id-2600496487/" driver$navigate(urll) # 处理cookie弹窗 Sys.sleep(2) # 等待页面加载 politicas <- driver$findElement(using = "css", value = "button.cookie-notifier__cta") politicas$clickElement() # 点击地图按钮打开模态框 Sys.sleep(1) botaomapa <- driver$findElement(using = "css", value = "button[aria-label='Ver mapa']") botaomapa$clickElement() # 切换到地图iframe Sys.sleep(2) # 等待模态框和iframe加载 iframe <- driver$findElement(using = "css", value = "iframe.listing-map__frame") driver$switchToFrame(iframe) # 定位并提取坐标文本 coord_element <- driver$findElement(using = "css", value = "div.mapboxgl-ctrl-geocoder--input") coordinates <- coord_element$getElementText()[[1]] print(coordinates) # 输出目标坐标:20º27'36.1"S 54º38'03.1"W # 清理资源 driver$close() rD$server$stop()
关键说明
- 使用相对选择器(如
button[aria-label='Ver mapa'])替代绝对XPATH,提升稳定性 - 加入
Sys.sleep()等待元素加载,避免因页面渲染延迟导致的定位失败 - 必须切换到iframe上下文才能访问其内部元素,这是动态页面抓取的常见要点
内容的提问来源于stack exchange,提问作者Ivan Bezerra Allaman
相关产品推荐
相关产品推荐

