Selenium以systemd服务运行时访问HTTPS网站超时问题求助

我开发了一个基于Selenium、Python和Flask的自动化项目，开发环境完全正常，但部署到生产环境以systemd服务运行时，Selenium访问网站（包括https://instagram.com）一直超时。奇怪的是，手动执行python app.py时功能完全正常，只有通过systemd服务运行才失败。systemd配置无特殊限制，相关Selenium函数代码如下：

@app.route("/automation-ticket/test-automation/run", methods=["POST"])
def run_test_automation():

    global last_result

    terminal_logs.clear()

    test_url = request.form.get(
        "test_url"
    )

    driver = None

    start = datetime.now()

    try:

        log_print("=" * 50)
        log_print("START TEST SELENIUM")
        log_print(f"START : {start}")

        env_info = f"""
USER = {os.getenv("USER")}
HOME = {os.getenv("HOME")}
PATH = {os.getenv("PATH")}
PWD  = {os.getcwd()}
"""

        log_print(env_info)

        options = webdriver.ChromeOptions()
        options.binary_location = "/home/anisa/browsers/chrome-linux64/chrome"

        options.set_capability("acceptInsecureCerts", True)
        options.add_argument("--headless=new")
        options.add_argument("--no-sandbox")
        options.add_argument("--disable-dev-shm-usage")
        options.add_argument("--disable-gpu")
        options.add_argument("--window-size=1920,1080")
        options.add_argument("--allow-insecure-localhost")
        options.add_argument("--ignore-certificate-errors")
        options.add_argument("--ignore-ssl-errors")
        options.add_argument("--ignore-certificate-errors")
        options.add_argument("--allow-running-insecure-content")
        options.add_argument("--disable-web-security")
        options.add_argument("--disable-features=CertificateTransparencyComponentUpdater")

        log_print("Starting ChromeDriver...")

        driver = webdriver.Chrome(
            service=Service("/usr/bin/chromedriver"),
            options=options
        )

        driver.set_page_load_timeout(30)

        log_print("ChromeDriver started successfully")

        goto_error = None

        try:
            log_print(test_url)
            log_print("Opening URL...")

            driver.get(test_url)

            log_print("driver.get SUCCESS")

        except Exception as e:

            goto_error = str(e)

            log_print("driver.get FAILED")
            log_print(goto_error)

            with open(
                "/home/anisa/page_timeout.txt",
                "w",
                encoding="utf-8"
            ) as f:
                f.write(goto_error)

        log_print("Waiting 20 seconds...")
        time.sleep(20)

        try:
            title = driver.title
            log_print(f"TITLE: {title}")
        except Exception as e:
            title = "N/A"
            log_print(f"TITLE ERROR: {e}")

        try:
            current_url = driver.current_url
            log_print(f"CURRENT URL: {current_url}")
        except Exception as e:
            current_url = "N/A"
            log_print(f"CURRENT URL ERROR: {e}")

        try:

            page_source = driver.page_source

            with open(
                "/home/anisa/page.html",
                "w",
                encoding="utf-8"
            ) as f:
                f.write(page_source)

            log_print(
                f"HTML SAVED ({len(page_source)} chars)"
            )

        except Exception as e:

            log_print(f"PAGE SOURCE ERROR: {e}")

            with open(
                "/home/anisa/page_source_error.txt",
                "w",
                encoding="utf-8"
            ) as f:
                f.write(str(e))

        try:

            ready_state = driver.execute_script(
                "return document.readyState"
            )

            log_print(
                f"READY STATE: {ready_state}"
            )

        except Exception as e:

            ready_state = f"ERROR: {e}"

            log_print(
                f"READY STATE ERROR: {e}"
            )

        try:

            body_length = len(driver.page_source)

        except:

            body_length = 0

        log_print(
            f"BODY LENGTH: {body_length}"
        )

        screenshot_path = os.path.join(
            app.root_path,
            "static",
            "test_systemd.png"
        )

        driver.save_screenshot(
            screenshot_path
        )

        log_print(
            f"SCREENSHOT SAVED: {screenshot_path}"
        )

        end = datetime.now()


        log_print(f"TARGET URL : {test_url}")
        log_print(f"END : {end}")
        log_print(f"DURATION : {end-start}")
        last_result = f"""
SUCCESS

TARGET URL:
{test_url}

TITLE:
{title}

CURRENT URL:
{current_url}

READY STATE:
{ready_state}

BODY LENGTH:
{body_length}

GOTO ERROR:
{goto_error}

SCREENSHOT:
{screenshot_path}
"""

    except Exception:

        error_ss = os.path.join(
            app.root_path,
            "static",
            "error_systemd.png"
        )

        log_print("EXCEPTION OCCURRED")
        log_print(traceback.format_exc())

        try:
            if driver:
                driver.save_screenshot(error_ss)
                log_print(
                    f"ERROR SCREENSHOT SAVED: {error_ss}"
                )
        except:
            pass

        last_result = traceback.format_exc()

    finally:

        try:
            if driver:
                driver.quit()
                log_print("ChromeDriver closed")
        except:
            pass

    return render_template(
        "test_selenium.html",
        result=last_result
    )


if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5000)
    #app.run(debug=True)

1. 统一systemd与手动执行的环境变量 #

systemd默认环境变量和用户会话不同，需在服务文件中显式配置：

• 编辑你的systemd服务文件（如/etc/systemd/system/your-service.service），在[Service]段添加：

  Environment="HOME=/home/anisa"
  Environment="PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/home/anisa/.local/bin:/home/anisa/browsers/chrome-linux64"
  

• 重新加载服务配置：sudo systemctl daemon-reload，重启服务：sudo systemctl restart your-service.service

2. 确保服务以正确用户运行 #

避免root用户运行Chrome带来的权限问题，在服务文件中指定普通用户：

[Service]
User=anisa
Group=anisa

同时检查Chrome二进制文件权限：

chmod +x /home/anisa/browsers/chrome-linux64/chrome

3. 优化Selenium超时与Chrome参数 #

• 延长页面加载超时时间：

  driver.set_page_load_timeout(60)
  

• 添加防检测与网络优化参数，替换原有的options配置部分：

  options = webdriver.ChromeOptions()
  options.binary_location = "/home/anisa/browsers/chrome-linux64/chrome"
  
  options.set_capability("acceptInsecureCerts", True)
  options.add_argument("--headless=new")
  options.add_argument("--no-sandbox")
  options.add_argument("--disable-dev-shm-usage")
  options.add_argument("--disable-gpu")
  options.add_argument("--window-size=1920,1080")
  options.add_argument("--disable-extensions")
  options.add_argument("--disable-plugins-discovery")
  options.add_argument("--disable-blink-features=AutomationControlled")
  options.add_experimental_option("excludeSwitches", ["enable-automation"])
  options.add_experimental_option('useAutomationExtension', False)
  

• 用显式等待替代固定sleep，确保页面加载完成：

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.support import expected_conditions as EC
  from selenium.webdriver.common.by import By
  
  # 替换原来的time.sleep(20)
  try:
      WebDriverWait(driver, 60).until(EC.presence_of_element_located((By.TAG_NAME, "body")))
      log_print("Page body loaded")
  except Exception as e:
      log_print(f"Wait for page load failed: {e}")
  

4. 排查systemd日志定位问题 #

实时查看服务运行日志，获取具体错误信息：

journalctl -u your-service-name.service -f

重点关注Chrome启动失败、DNS解析错误、网络连接超时等日志内容。

5. 验证服务上下文的网络连通性 #

在服务中添加网络测试步骤，确认systemd环境下能正常访问目标网站：

import subprocess

# 在启动Chrome前添加
try:
    result = subprocess.run(["curl", "-I", test_url], capture_output=True, text=True, timeout=30)
    log_print(f"Curl test result:\n{result.stdout}\n{result.stderr}")
except subprocess.TimeoutExpired:
    log_print("Curl test timed out")

内容的提问来源于stack exchange，提问作者Fajar Lazuardi