Python Selenium双实例同时运行触发ConnectionResetError问题求助

问题情况 #

• 两个执行不同操作的Python Selenium脚本，单独运行数小时均无异常
• 同时运行时，不到一小时就会触发ConnectionResetError: [WinError 10054]，该错误仅在同时运行场景下出现
• 两个脚本使用同一ChromeDriver二进制文件

疑问与怀疑方向 #

1. 是否可通过修改Selenium WebDriver的配置或选项，避免两个实例间的干扰？
2. 怀疑错误并非源于Selenium实例干扰，而是两个独立web hook的发送操作导致：错误栈中唯一指向自定义代码的行是hook.send(embed=embed)（位于C:\Users\User\Documents\python\waifu\new_waifus.py第101行）

配置详情 #

• Chrome版本：104.0.5112.102
• ChromeDriver版本：104.0.5112.20（已尽量匹配Chrome版本）
• 操作系统：Windows 10 Pro
• Python版本：3.9.13
• Selenium版本：4.4.3

完整错误日志 #

Traceback (most recent call last):                                                                                                                                                                                                             File "C:\Users\User\AppData\Roaming\Python\Python39\site-packages\urllib3\connectionpool.py", line 703, in urlopen                                                                                                                             httplib_response = self._make_request(                                                                                                                                                                                                     File "C:\Users\User\AppData\Roaming\Python\Python39\site-packages\urllib3\connectionpool.py", line 449, in _make_request                                                                                                                       six.raise_from(e, None)                                                                                                                                                                                                                    File "<string>", line 3, in raise_from                                                                                                                                                                                                       File "C:\Users\User\AppData\Roaming\Python\Python39\site-packages\urllib3\connectionpool.py", line 444, in _make_request                                                                                                                       httplib_response = conn.getresponse()                                                                                                                                                                                                      File "C:\Program Files\Python39\lib\http\client.py", line 1377, in getresponse                                                                                                                                                                 response.begin()                                                                                                                                                                                                                           File "C:\Program Files\Python39\lib\http\client.py", line 320, in begin                                                                                                                                                                        version, status, reason = self._read_status()                                                                                                                                                                                              File "C:\Program Files\Python39\lib\http\client.py", line 281, in _read_status                                                                                                                                                                 line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")                                                                                                                                                                                   File "C:\Program Files\Python39\lib\socket.py", line 704, in readinto                                                                                                                                                                          return self._sock.recv_into(b)                                                                                                                                                                                                             File "C:\Program Files\Python39\lib\ssl.py", line 1242, in recv_into                                                                                                                                                                           return self.read(nbytes, buffer)                                                                                                                                                                                                           File "C:\Program Files\Python39\lib\ssl.py", line 1100, in read                                                                                                                                                                                return self._sslobj.read(len, buffer)                                                                                                                                                                                                    ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host                                                                                                                                                                                                                                                                                                                                                                                      During handling of the above exception, another exception occurred:                                                                                                                                                                                                                                                                                                                                                                                                                       Traceback (most recent call last):                                                                                                                                                                                                             File "C:\Users\User\AppData\Roaming\Python\Python39\site-packages\requests\adapters.py", line 489, in send                                                                                                                                     resp = conn.urlopen(                                                                                                                                                                                                                       File "C:\Users\User\AppData\Roaming\Python\Python39\site-packages\urllib3\connectionpool.py", line 787, in urlopen                                                                                                                             retries = retries.increment(                                                                                                                                                                                                               File "C:\Users\User\AppData\Roaming\Python\Python39\site-packages\urllib3\util\retry.py", line 550, in increment                                                                                                                               raise six.reraise(type(error), error, _stacktrace)                                                                                                                                                                                         File "C:\Users\User\AppData\Roaming\Python\Python39\site-packages\urllib3\packages\six.py", line 769, in reraise                                                                                                                               raise value.with_traceback(tb)                                                                                                                                                                                                             File "C:\Users\User\AppData\Roaming\Python\Python39\site-packages\urllib3\connectionpool.py", line 703, in urlopen                                                                                                                             httplib_response = self._make_request(                                                                                                                                                                                                     File "C:\Users\User\AppData\Roaming\Python\Python39\site-packages\urllib3\connectionpool.py", line 449, in _make_request                                                                                                                       six.raise_from(e, None)                                                                                                                                                                                                                    File "<string>", line 3, in raise_from                                                                                                                                                                                                       File "C:\Users\User\AppData\Roaming\Python\Python39\site-packages\urllib3\connectionpool.py", line 444, in _make_request                                                                                                                       httplib_response = conn.getresponse()                                                                                                                                                                                                      File "C:\Program Files\Python39\lib\http\client.py", line 1377, in getresponse                                                                                                                                                                 response.begin()                                                                                                                                                                                                                           File "C:\Program Files\Python39\lib\http\client.py", line 320, in begin                                                                                                                                                                        version, status, reason = self._read_status()                                                                                                                                                                                              File "C:\Program Files\Python39\lib\http\client.py", line 281, in _read_status                                                                                                                                                                 line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")                                                                                                                                                                                   File "C:\Program Files\Python39\lib\socket.py", line 704, in readinto                                                                                                                                                                          return self._sock.recv_into(b)                                                                                                                                                                                                             File "C:\Program Files\Python39\lib\ssl.py", line 1242, in recv_into                                                                                                                                                                           return self.read(nbytes, buffer)                                                                                                                                                                                                           File "C:\Program Files\Python39\lib\ssl.py", line 1100, in read                                                                                                                                                                                return self._sslobj.read(len, buffer)                                                                                                                                                                                                    urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))                                                                                                                                                                                                                                                                                                                During handling of the above exception, another exception occurred:                                                                                                                                                                                                                                                                                                                                                                                                                       Traceback (most recent call last):                                                                                                                                                                                                             File "C:\Users\User\Documents\python\waifu\new_waifus.py", line 101, in <module>                                                                                                                                                               hook.send(embed=embed)                                                                                                                                                                                                                     File "C:\Users\User\AppData\Roaming\Python\Python39\site-packages\dhooks\client.py", line 275, in send                                                                                                                                         return self._request('POST', payload, file=file)                                                                                                                                                                                           File "C:\Users\User\AppData\Roaming\Python\Python39\site-packages\dhooks\client.py", line 359, in _request                                                                                                                                     resp = self.session.post(self.url, json=payload,                                                                                                                                                                                           File "C:\Users\User\AppData\Roaming\Python\Python39\site-packages\requests\sessions.py", line 635, in post                                                                                                                                     return self.request("POST", url, data=data, json=json, **kwargs)                                                                                                                                                                           File "C:\Users\User\AppData\Roaming\Python\Python39\site-packages\requests\sessions.py", line 587, in request                                                                                                                                  resp = self.send(prep, **send_kwargs)                                                                                                                                                                                                      File "C:\Users\User\AppData\Roaming\Python\Python39\site-packages\requests\sessions.py", line 701, in send                                                                                                                                     r = adapter.send(request, **kwargs)                                                                                                                                                                                                        File "C:\Users\User\AppData\Roaming\Python\Python39\site-packages\requests\adapters.py", line 547, in send                                                                                                                                     raise ConnectionError(err, request=request)                                                                                                                                                                                              requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))

排查建议 #

针对Selenium实例隔离的优化 #

1. 为每个WebDriver实例分配独立的用户数据目录，避免共享资源冲突

   from selenium import webdriver
   from selenium.webdriver.chrome.options import Options
   
   # 脚本1使用的配置
   options1 = Options()
   options1.add_argument(r"--user-data-dir=C:\Users\User\Documents\chrome_profile_script1")
   driver1 = webdriver.Chrome(options=options1)
   
   # 脚本2使用的配置（路径不同）
   options2 = Options()
   options2.add_argument(r"--user-data-dir=C:\Users\User\Documents\chrome_profile_script2")
   driver2 = webdriver.Chrome(options=options2)
   

2. 添加资源释放保障，确保脚本结束时彻底关闭WebDriver

   try:
       # 你的脚本业务逻辑
   finally:
       driver.quit()  # 使用quit()而非close()，确保销毁所有关联进程
   

3. 启用无头模式并禁用共享内存（如果不需要可视化界面）

   options.add_argument("--headless=new")
   options.add_argument("--disable-dev-shm-usage")
   

针对web hook连接错误的优化 #

1. 为hook请求添加重试机制，处理临时网络或服务器限流问题

   from requests.adapters import HTTPAdapter
   from urllib3.util.retry import Retry
   import requests
   from dhooks import Webhook
   
   # 创建带重试的session
   session = requests.Session()
   retry_strategy = Retry(
       total=3,
       backoff_factor=1,
       status_forcelist=[429, 500, 502, 503, 504]
   )
   adapter = HTTPAdapter(max_retries=retry_strategy)
   session.mount("https://", adapter)
   session.mount("http://", adapter)
   
   # 给hook绑定这个session
   hook = Webhook("你的hook_url")
   hook.session = session
   

2. 控制hook请求频率，避免并发过高触发服务器拦截
   可以在发送请求前添加短暂延迟：

   import time
   time.sleep(0.5)  # 延迟0.5秒再发送
   hook.send(embed=embed)
   

3. 确保两个脚本使用独立的requests Session实例，避免连接池共享导致的冲突

内容的提问来源于stack exchange，提问作者Yeet Lmao