You need to enable JavaScript to run this app.
优惠活动
大模型
产品
解决方案
定价
更多

Linux环境下R selenider爬虫遇--user-data-dir错误求助

问题描述

我尝试使用以下R代码进行网页爬取:

library(selenider)
library(rvest)

session <- selenider_session("selenium", browser = "chrome")
Sys.sleep(3)

open_url("https://egamersworld.com/callofduty/matches")

elements <- session |> get_page_source() |> html_elements(".item_teams__cKXQT")

res <- data.frame(
  home_team_name = elements |> 
    html_elements(".item_team__evhUQ:nth-child(1) .item_teamName__NSnfH") |> 
    html_text(trim = TRUE),
  home_team_odds = elements |> 
    html_elements(".item_team__evhUQ:nth-child(1) .item_odd__Lm2Wl") |> 
    html_text(trim = TRUE),
  away_team_name = elements |> 
    html_elements(".item_team__evhUQ:nth-child(3) .item_teamName__NSnfH") |> 
    html_text(trim = TRUE),
  away_team_odds = elements |> 
    html_elements(".item_team__evhUQ:nth-child(3) .item_odd__Lm2Wl") |> 
    html_text(trim = TRUE),
  match_date = elements |> 
    html_elements(".item_scores__Vi7YX .item_date__g4cq_") |> 
    html_text(trim = TRUE),
  match_time = elements |> 
    html_elements(".item_scores__Vi7YX .item_time__xBia_") |> 
    html_text(trim = TRUE),
  match_type = elements |> 
    html_elements(".item_scores__Vi7YX .item_bo__u2C9Q") |> 
    html_text(trim = TRUE)
)

该代码在Windows 10本地运行正常,但部署到Linux服务器时出现如下错误:

Error in `create_selenium_client_internal()`:
! A Selenium session could not be started
Caused by error in `httr2::req_perform()`:
! HTTP 500 Internal Server Error.
✖ Session not created.
✖ Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: probably user data directory is already in use, please specify a unique value for --user-data-dir argument, or don't use --user-data-dir 
  Host info: host: 'Unknown', ip: 'Unknown'
  Build info: version: '4.29.0', revision: '18ae989'
  System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-134-generic', java.version: '11.0.26'
  Driver info: driver.version: unknown
  Build info: version: '4.29.0', revision: '18ae989'
  System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-134-generic', java.version: '11.0.26'
  Driver info: driver.version: unknown

我尝试手动创建并指定目录,但仍出现相同错误:

server_options = selenium_options(server_options = selenium_server_options(extra_args = c("--user-data-dir=/tmp/testing")))

session <- selenider_session(
  "selenium", 
  browser = "chrome",
  options = server_options
)

我也尝试杀死所有Chrome进程,但无济于事。服务器上有其他Python Selenium脚本可正常运行,且未手动设置--user-data-dir参数,我更擅长R,希望将代码迁移到R环境。

解决方案

针对Linux服务器上的这个问题,可尝试以下几种方法:

1. 配置Chrome生成唯一临时目录并启用无头模式

直接在Chrome选项中生成基于时间戳的唯一用户数据目录,同时启用无头模式适配服务器环境,附加Linux运行必备参数:

library(selenider)
library(rvest)

# 配置Chrome选项
chrome_opts <- chrome_options(
  extra_args = c(
    "--headless=new",
    "--no-sandbox",
    "--disable-dev-shm-usage",
    "--user-data-dir=/tmp/selenium-chrome-$(date +%s)"
  )
)

# 创建会话
session <- selenider_session(
  "selenium", 
  browser = "chrome",
  options = selenium_options(browser_options = chrome_opts)
)

# 后续爬取代码保持不变
Sys.sleep(3)
open_url("https://egamersworld.com/callofduty/matches")

elements <- session |> get_page_source() |> html_elements(".item_teams__cKXQT")

res <- data.frame(
  home_team_name = elements |> 
    html_elements(".item_team__evhUQ:nth-child(1) .item_teamName__NSnfH") |> 
    html_text(trim = TRUE),
  home_team_odds = elements |> 
    html_elements(".item_team__evhUQ:nth-child(1) .item_odd__Lm2Wl") |> 
    html_text(trim = TRUE),
  away_team_name = elements |> 
    html_elements(".item_team__evhUQ:nth-child(3) .item_teamName__NSnfH") |> 
    html_text(trim = TRUE),
  away_team_odds = elements |> 
    html_elements(".item_team__evhUQ:nth-child(3) .item_odd__Lm2Wl") |> 
    html_text(trim = TRUE),
  match_date = elements |> 
    html_elements(".item_scores__Vi7YX .item_date__g4cq_") |> 
    html_text(trim = TRUE),
  match_time = elements |> 
    html_elements(".item_scores__Vi7YX .item_time__xBia_") |> 
    html_text(trim = TRUE),
  match_type = elements |> 
    html_elements(".item_scores__Vi7YX .item_bo__u2C9Q") |> 
    html_text(trim = TRUE)
)

# 爬取完成后显式关闭会话释放资源
close_session(session)

2. 检查并匹配Selenium与ChromeDriver版本

确保服务器上的ChromeDriver版本与安装的Chrome浏览器版本完全兼容,Selenium 4.29.0需要对应版本的ChromeDriver。可通过以下命令检查Chrome版本:

google-chrome --version

也可使用wdman包自动管理驱动版本:

library(wdman)
chrome() # 自动安装匹配的ChromeDriver

3. 用Docker封装运行环境(可选)

若服务器环境复杂,可使用预配置的Selenium Docker镜像避免依赖问题:

docker run -d -p 4444:4444 selenium/standalone-chrome:latest

然后在R代码中连接远程Selenium服务器:

session <- selenider_session(
  "remote",
  browser = "chrome",
  server_url = "http://localhost:4444/wd/hub"
)

内容的提问来源于stack exchange,提问作者Nick Amato

相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级,最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分), 限时 9.9 元起

最近更新时间:2026.06.13 23:45:08