OpenShift Pod中Firefox Selenium WebDriver初始化超时问题排查

问题描述 #

在OpenShift Pod中运行Python 3.9结合Selenium调用无头模式Firefox时，WebDriver初始化阶段触发超时错误，连接geckodriver进程在默认120秒等待后超时，尚未进入页面导航阶段。

环境详情 #

• 运行环境：OpenShift Pod
• Python版本：3.9
• Selenium版本：最新版
• 浏览器：无头模式Firefox
• 网络：使用企业代理（已设置HTTP_PROXY环境变量）

已尝试操作 #

• 添加容器化适配参数：--no-sandbox、--disable-dev-shm-usage、--disable-gpu
• 确认geckodriver已安装在指定路径
• 代理配置在其他网络操作中正常工作

错误日志 #

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 468, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 463, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/lib/python3.9/http/client.py", line 1377, in getresponse
    response.begin()
  File "/usr/local/lib/python3.9/http/client.py", line 320, in begin
    version, status, reason = self._read_status()
  File "/usr/local/lib/python3.9/http/client.py", line 281, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/local/lib/python3.9/socket.py", line 716, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/seleniumCapture.py", line 28, in <module>
    driver = webdriver.Firefox(options=firefox_options, service=service)
  File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/firefox/webdriver.py", line 71, in __init__
    super().__init__(command_executor=executor, options=options)
  File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 250, in __init__
    self.start_session(capabilities)
  File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 342, in start_session
    response = self.execute(Command.NEW_SESSION, caps)["value"]
  File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 427, in execute
    response = self.command_executor.execute(driver_command, params)
  File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/remote_connection.py", line 404, in execute
    return self._request(command_info[0], url, body=data)
  File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/remote_connection.py", line 428, in _request
    response = self._conn.request(method, url, body=body, headers=headers, timeout=self._client_config.timeout)
  File "/usr/local/lib/python3.9/site-packages/urllib3/request.py", line 81, in request
    return self.request_encode_body(
  File "/usr/local/lib/python3.9/site-packages/urllib3/request.py", line 173, in request_encode_body
    return self.urlopen(method, url, **extra_kw)
  File "/usr/local/lib/python3.9/site-packages/urllib3/poolmanager.py", line 376, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 802, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.9/site-packages/urllib3/util/retry.py", line 552, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.9/site-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 716, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 470, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 358, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='localhost', port=50597): Read timed out. (read timeout=120)

代码片段 #

from selenium.webdriver.firefox.options import Options as FirefoxOptions
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.keys import Keys
from selenium import webdriver
from selenium.webdriver.firefox.service import Service
import time
import urllib3
import logging
import os

# Init
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')

firefox_options = FirefoxOptions()
firefox_options.add_argument("--headless")
firefox_options.add_argument("--no-sandbox")
firefox_options.add_argument("--disable-dev-shm-usage")
firefox_options.add_argument("--disable-gpu")

#geckodriver doit être installé et disponible dans le PATH
gecko_path = os.environ.get("GECKO_DRIVER_PATH", "/usr/local/bin/geckodriver")
service = Service(executable_path=gecko_path)

driver = webdriver.Firefox(options=firefox_options, service=service)

def connect_to_cacti():
    try:
        #proxy OpenShift
        proxy_env = os.environ.get("HTTP_PROXY")
        if proxy_env:
            logging.info(f"Utilisation du proxy: {proxy_env}")
            firefox_options.set_preference("network.proxy.type", 1)
            proxy_components = proxy_env.replace("http://", "").split(":")
            firefox_options.set_preference("network.proxy.http", proxy_components[0])
            firefox_options.set_preference("network.proxy.http_port", int(proxy_components[1]))
            firefox_options.set_preference("network.proxy.ssl", proxy_components[0])
            firefox_options.set_preference("network.proxy.ssl_port", int(proxy_components[1]))

        driver.get('https://cacti-sma-ocb.securite.fr.intraorange/cacti/index.php')

        wait = WebDriverWait(driver, 20)

        login_field = wait.until(
            EC.presence_of_element_located((By.NAME, "login_username"))
        )
        password_field = wait.until(
            EC.presence_of_element_located((By.NAME, "login_password"))
        )

        login_field.send_keys('*******')
        password_field.send_keys('********')

        password_field.send_keys(Keys.RETURN)
        time.sleep(5)

    except Exception as e:
        logging.error(f"Une erreur s'est produite lors de la connexion à Cacti: {str(e)}")
        raise

    finally:
        logging.debug("Tentative de connexion à Cacti terminée")

def get_widget(url):
    try:
        driver.get(url)
        # Augmenter le temps d'attente pour s'assurer que la page se charge complètement
        time.sleep(5)
        driver.save_screenshot("photo.png")
        logging.info("Capture d'écran prise avec succès")
    except Exception as e:
        logging.error(f"Erreur lors de la capture du widget: {str(e)}")
        raise
    finally:
        True

def cleanup():
    if driver:
        driver.quit()

# Main
if __name__ == "__main__":
    try:
        connect_to_cacti()
        # Exemple d'appel à get_widget
        # get_widget('http://cacti-sma-ocb.securite.fr.intraorange/cacti/graph.php?action=zoom&local_graph_id=1533938&rra_id=0&view_type=tree&graph_start=1733953560&graph_end=1741729560')
    finally:
        cleanup()

可能的原因与解决方案 #

1. 代理配置时机错误 #

代码在初始化driver之后才设置Firefox代理偏好，导致配置不生效，且代理规则可能拦截geckodriver与localhost的通信。

修复方法：在初始化driver前完成所有Firefox偏好设置，并排除本地地址的代理：

# 先处理代理配置
proxy_env = os.environ.get("HTTP_PROXY")
if proxy_env:
    logging.info(f"Utilisation du proxy: {proxy_env}")
    firefox_options.set_preference("network.proxy.type", 1)
    proxy_components = proxy_env.replace("http://", "").split(":")
    firefox_options.set_preference("network.proxy.http", proxy_components[0])
    firefox_options.set_preference("network.proxy.http_port", int(proxy_components[1]))
    firefox_options.set_preference("network.proxy.ssl", proxy_components[0])
    firefox_options.set_preference("network.proxy.ssl_port", int(proxy_components[1]))
    # 关键：排除localhost，避免geckodriver连接被代理拦截
    firefox_options.set_preference("network.proxy.no_proxies_on", "localhost,127.0.0.1")

# 再初始化driver
driver = webdriver.Firefox(options=firefox_options, service=service)

2. OpenShift资源限制导致启动缓慢 #

Pod的CPU或内存资源不足，会拖慢Firefox和geckodriver的启动速度，触发超时。

修复方法：

• 调整Pod资源请求与限制，例如设置CPU请求500m、限制1核，内存请求1Gi、限制2Gi
• 添加窗口大小参数，避免Firefox因渲染问题卡顿：

  firefox_options.add_argument("--width=1920")
  firefox_options.add_argument("--height=1080")
  

3. Geckodriver与Firefox版本不兼容 #

最新版Selenium需要特定版本的geckodriver与Firefox匹配，版本不匹配会导致初始化失败。

修复方法：

• 对照Mozilla官方版本对应表，确保geckodriver与Firefox版本完全匹配
• 在Dockerfile中明确指定Firefox和geckodriver的安装版本，避免自动更新导致不兼容

4. OpenShift安全策略限制 #

OpenShift的安全上下文约束（SCC）可能限制容器内进程权限，导致Firefox无法正常启动。

修复方法：

• 为Pod配置allowPrivilegeEscalation: true，或使用anyuid SCC（需评估安全风险）
• 在Dockerfile中以非root用户运行，并确保该用户拥有geckodriver和Firefox目录的访问权限

5. 延长初始化超时时间 #

默认120秒超时在资源受限的OpenShift环境中可能不足。

修复方法：

service = Service(executable_path=gecko_path)
service.start_timeout = 300  # 设置为5分钟

driver = webdriver.Firefox(options=firefox_options, service=service)
driver.set_page_load_timeout(300)

内容的提问来源于stack exchange，提问作者N7Legend