Selenium XVFB运行报错:Unable to receive message from renderer
问题概述
Selenium 爬虫程序在headless模式下可完美运行,单独通过XVFB启动虚拟显示器也无任何报错,测试代码如下:
from xvfbwrapper import Xvfb vdisplay = Xvfb() vdisplay.start() vdisplay.stop()
但当尝试将二者结合运行时,程序抛出如下错误:
[ERROR] SessionNotCreatedException: Message: session not created
from disconnected: Unable to receive message from renderer (Session info: chrome=96.0.4664.0)
报错堆栈
Traceback (most recent call last): File "/var/task/slack_main.py", line 34, in handler scrape_price(asin_list) File "/var/task/slack_main.py", line 58, in scrape_price driver = webdriver.Chrome("/opt/chromedriver",options=options) File "/var/lang/lib/python3.9/site-packages/selenium/webdriver/chrome/webdriver.py", line 70, in __init__ super(WebDriver, self).__init__(DesiredCapabilities.CHROME['browserName'], "goog", File "/var/lang/lib/python3.9/site-packages/selenium/webdriver/chromium/webdriver.py", line 92, in __init__ RemoteWebDriver.__init__( File "/var/lang/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 275, in __init__ self.start_session(capabilities, browser_profile) File "/var/lang/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 365, in start_session response = self.execute(Command.NEW_SESSION, parameters) File "/var/lang/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 430, in execute self.error_handler.check_response(response) File "/var/lang/lib/python3.9/site-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response raise exception_class(message, screen, stacktrace)
当前配置
以下为完整的Selenium与XVFB配置代码:
from selenium import webdriver from selenium_stealth import stealth from xvfbwrapper import Xvfb vdisplay = Xvfb() vdisplay.start() options = webdriver.ChromeOptions() prefs = {"browser.downloads.dir": "//tmp//", "download.default_directory": "//tmp//", "directory_upgrade": True} options.add_experimental_option("prefs", prefs) options.binary_location = '/opt/chrome/chrome' #options.add_argument('--headless') # 切换XVFB运行模式时手动开关该参数 options.add_argument('--no-sandbox') options.add_argument("--disable-gpu") options.add_argument("--window-size=1280x1696") options.add_argument("--single-process") options.add_argument("--disable-dev-shm-usage") options.add_argument("--disable-dev-tools") options.add_argument("--no-zygote") options.set_capability('unhandledPromptBehavior', 'ignore') options.add_argument("download.default_directory=/tmp") driver = webdriver.Chrome("/opt/chromedriver",options=options) stealth(driver, languages=["en-US", "en"], vendor="Google Inc.", platform="Win32", fix_hairline=True, ) vdisplay.stop() driver.close()
为何程序无法连接到显示器? 个人猜测该问题是否与--headless参数的开关有关?
环境版本信息
- Selenium 版本 3.141.0
- xvfbwrapper 版本 0.2.9
- 使用Docker编译项目并推送至AWS Lambda部署,采用公开的docker-selenium-lambda基础镜像 (无论是否启用XVFB,均未修改Dockerfile配置)
补充说明
已在所用基础镜像的公开代码仓库中找到一份XVFB配置相关的参考实现,完全采用该实现中的代码运行仍出现相同报错,怀疑问题是否与AWS Lambda的底层环境有关?
内容的提问来源于stack exchange,提问作者Luke Hamilton
相关产品推荐
相关产品推荐

