SeleniumBase在Linux环境下无法通过网站检测的问题排查

环境差异 #

• 正常运行：Mac（PyCharm + Selenium/SeleniumBase + undetected-chromedriver）、Windows 11，登录后可跳转至账户详情页或MFA验证页
• 异常情况：Ubuntu 23.10、Alpine 3.18，点击登录按钮后跳转至“出现问题”页面

关键版本信息 #

Python 3.11.6
selenium             4.16.0
seleniumbase         4.22.0

chromedriver --version
ChromeDriver 120.0.6099.71 (9729082fe6174c0a371fc66501f5efc5d69d3d2b-refs/branch-heads/6099_56@{#13})

google-chrome --version
Google Chrome 120.0.6099.71 

已尝试的测试代码 #

基础Driver方式 #

from seleniumbase import Driver
driver = Driver(uc=True, headless2=True)
driver.get("https://www.originenergy.com.au/my")
print("UA= ", driver.get_user_agent())

driver.type('//*[@id="login_username"]', "email_address")
driver.type('//*[@id="password"]', "password")
driver.click('//*[@id="root"]/div/div/div/div/div/div[2]/form/div/button')

SB上下文管理器方式 #

from seleniumbase import SB

with SB(uc=True, headless=True, xvfb=True) as sb:
        sb.open("https://www.originenergy.com.au/my")
        print("UA: ", sb.get_user_agent())
        print("\npage source\n", sb.get_page_source())
        sb.type('//*[@id="login_username"]', "email_address")
        sb.type('//*[@id="password"]', "password")
        print("\nUsername and Password entered\n")
        sb.click('//*[@id="root"]/div/div/div/div/div/div[2]/form/div/button')
        sb.sleep(10)

带用户配置文件的尝试 #

driver = Driver(uc=True, headless2=True, user_data_dir="/script/datadir")

更新信息：极简Linux环境测试 #

重建环境包含：Python3.11、venv、pip、seleniumbase、x11vnc、xvfb、google-chrome

测试代码 #

from seleniumbase import Driver
driver = Driver(uc=True, headed=True, user_data_dir="/script/newdir")
driver.get("https://www.originenergy.com.au/my")
print("UA= ", driver.get_user_agent())

driver.type('//*[@id="login_username"]', "email_address")
driver.type('//*[@id="password"]', "password")
driver.click('//*[@id="root"]/div/div/div/div/div/div[2]/form/div/button')

测试结果 #

• 控制台输出UA：Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36
• Chrome窗口正常打开，用户名密码输入正常，DevTools无“undetected-chromedriver 1337!”信息
• 仍跳转至错误页面，DevTools提前出现Kasada SDK相关429错误：

  Failed to load resource: the server responded with a status of 429 () - /149e9513-01fa-4fb0-…x-kpsdk-v=j-0.0.0:1 
  

• 最终错误日志：

  [error] handleError invoked myemail@gmail.com with errorCode: 429 and uniqueId: N/A Error: myemail@gmail.com:  errorCode: 429, uniqueId: N/A
      at Ue (LoginErrorHandler.ts:15:7)
      at r.handleError (withLogin.tsx:104:61)
      at withLogin.tsx:140:14
      at s (runtime.js:63:15)
      at Generator._invoke (runtime.js:293:1)
      at Generator.throw (runtime.js:118:1)
      at i (asyncToGenerator.js:5:1)
      at u (asyncToGenerator.js:31:1)
      at nrWrapper (login?state=hKFo2SBT…Mi4yIn0%3D:56:18441)
  r   @   fs.js:4
  p   @   sumoLogger.js:10
  error   @   index.js:2
  Ue  @   LoginErrorHandler.ts:13
  (anonymous) @   withLogin.tsx:104
  (anonymous) @   withLogin.tsx:140
  s   @   runtime.js:63
  (anonymous) @   runtime.js:293
  (anonymous) @   runtime.js:118
  i   @   asyncToGenerator.js:5
  u   @   asyncToGenerator.js:31
  nrWrapper   @   login?state=hKFo2SBT…oiOS4xMi4yIn0%3D:56
  Promise.then (async)        
  nrWrapper   @   login?state=hKFo2SBT…oiOS4xMi4yIn0%3D:56
  i   @   asyncToGenerator.js:15
  c   @   asyncToGenerator.js:27
  (anonymous) @   asyncToGenerator.js:34
  nrWrapper   @   login?state=hKFo2SBT…oiOS4xMi4yIn0%3D:56
  r   @   login?state=hKFo2SBT…oiOS4xMi4yIn0%3D:56
  t   @   _export.js:36
  (anonymous) @   asyncToGenerator.js:23
  (anonymous) @   withLogin.tsx:21
  m   @   react-dom.production.min.js:15
  w   @   react-dom.production.min.js:15
  (anonymous) @   react-dom.production.min.js:16
  S   @   react-dom.production.min.js:16
  C   @   react-dom.production.min.js:17
  k   @   react-dom.production.min.js:17
  N   @   react-dom.production.min.js:17
  Rn  @   react-dom.production.min.js:85
  Fn  @   react-dom.production.min.js:87
  t.unstable_runWithPriority  @   scheduler.production.min.js:20
  ho  @   react-dom.production.min.js:113
  yc  @   react-dom.production.min.js:207
  Dn  @   react-dom.production.min.js:86
  nrWrapper   @   login?state=hKFo2SBT…oiOS4xMi4yIn0%3D:56
  

解决建议 #

1. 模拟真实用户行为细节

   • 输入字符时用slow_type替代type，模拟真实打字速度
   • 在页面加载、输入操作、点击按钮前添加1-3秒随机延迟，避免机械性操作
   • 点击按钮前先执行hover操作，模拟用户鼠标移动过程

2. 优化浏览器指纹参数

   • 添加更多浏览器启动参数，贴近真实Linux桌面环境：

     driver = Driver(
         uc=True,
         headed=True,
         user_data_dir="/script/newdir",
         args=[
             "--start-maximized",
             "--disable-blink-features=AutomationControlled",
             "--no-sandbox",
             "--disable-dev-shm-usage",
             "--disable-features=VizDisplayCompositor"
         ]
     )
     

   • 自定义UA，确保和真实Linux桌面Chrome的UA完全一致

3. 针对Kasada验证的调整

   • 使用SeleniumBase的uc_cdp命令注入指纹参数，修改navigator相关属性，缩小和真实浏览器的差异
   • 尝试禁用Kasada相关脚本加载（注意可能导致页面功能异常）

4. 网络与环境优化

   • 检查Linux服务器IP是否被标记为异常，必要时更换IP或使用代理
   • 同步Linux环境的时区、语言设置为真实用户常用值

5. 版本更新

   • 升级SeleniumBase到最新版本，新版本可能修复了Linux环境下的指纹伪装问题

内容的提问来源于stack exchange，提问作者davesc63