求助：Playwright同步API多独立脚本并发运行失败问题

问题背景 #

• 多个独立脚本使用Playwright同步API，分时运行一切正常；同时运行时，先启动的脚本会占用浏览器，后续脚本抛出Browser closed错误，日志显示Opening in existing browser session
• 需要通过持久化浏览器上下文保留用户认证状态，但共用同一用户数据目录是冲突的核心原因
• 寻求无需切换到异步API的并发运行方案

原启动代码 #

import os
from playwright.sync_api import Playwright, sync_playwright


def get_cookies(context):
    """
    Checks for new cookies. This function running is critical to pages loading 
    properly, or else the pages will look like they are only partially loading.
    """
    global cookies
    cookie_set = {tempdict['name'] for tempdict in cookies}
    new_cookies = context.cookies()
    context_cookie_set = {tempdict['name'] for tempdict in new_cookies}
    new_cookie_set = context_cookie_set - cookie_set
    if len(new_cookie_set) == 0:
        return 0
    else:
        pass

    cookies = new_cookies
    return len(new_cookie_set)



cookies = []
playwright = sync_playwright().start()
username = os.environ.get('USERNAME')
context_location = rf'C:\Users\{username}\AppData\Local\Microsoft\Edge\User Data\Default'
context = playwright.chromium.launch_persistent_context(context_location, 
                                                          headless=False, 
                                                                channel='msedge')
get_cookies(context)
context.on('requestfinished', lambda r: get_cookies(context))
context.on('requestfailed', lambda r: get_cookies(context))
page = context.new_page()
page.goto('https://playwright.dev/python/')
page.wait_for_load_state(state='networkidle')
while get_cookies(context) > 0:
    print('.')
    pass
# Simulate an action with a time delay
seconds = 30
page.wait_for_timeout(seconds * 1000)

报错信息 #

Traceback (most recent call last):
  File "E:\OLD_STUFF\Miscellaneous_Scripts\playwright_test.py", line 28, in <module>
    context = playwright.chromium.launch_persistent_context(context_location,
  File "c:\programdata\miniconda3\lib\site-packages\playwright\sync_api\_generated.py", line 14662, in launch_persistent_context
    self._sync(
  File "c:\programdata\miniconda3\lib\site-packages\playwright\_impl\_sync_base.py", line 104, in _sync
    return task.result()
  File "c:\programdata\miniconda3\lib\site-packages\playwright\_impl\_browser_type.py", line 155, in launch_persistent_context
    from_channel(await self._channel.send("launchPersistentContext", params)),
  File "c:\programdata\miniconda3\lib\site-packages\playwright\_impl\_connection.py", line 44, in send
    return await self._connection.wrap_api_call(
  File "c:\programdata\miniconda3\lib\site-packages\playwright\_impl\_connection.py", line 419, in wrap_api_call
    return await cb()
  File "c:\programdata\miniconda3\lib\site-packages\playwright\_impl\_connection.py", line 79, in inner_send
    result = next(iter(done)).result()
playwright._impl._api_types.Error: Browser closed.
==================== Browser output: ====================
<launching> C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe --disable-field-trial-config --disable-background-networking --enable-features=NetworkService,NetworkServiceInProcess --disable-background-timer-throttling --disable-backgrounding-occluded-windows --disable-back-forward-cache --disable-breakpad --disable-client-side-phishing-detection --disable-component-extensions-with-background-pages --disable-component-update --no-default-browser-check --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=ImprovedCookieControls,LazyFrameLoading,GlobalMediaControls,DestroyProfileOnBrowserClose,MediaRouter,DialMediaRouteProvider,AcceptCHFrame,AutoExpandDetailsElement,CertificateTransparencyComponentUpdater,AvoidUnnecessaryBeforeUnloadCheckSync,Translate --allow-pre-commit-input --disable-hang-monitor --disable-ipc-flooding-protection --disable-popup-blocking --disable-prompt-on-repost --disable-renderer-backgrounding --disable-sync --force-color-profile=srgb --metrics-recording-only --no-first-run --enable-automation --password-store=basic --use-mock-keychain --no-service-autorun --export-tagged-pdf --no-sandbox --user-data-dir=C:\Users\****\AppData\Local\Microsoft\Edge\User Data\Default --remote-debugging-pipe about:blank
<launched> pid=529300
[pid=529300][out] Opening in existing browser session.
[pid=529300] <process did exit: exitCode=0, signal=null>
[pid=529300] starting temporary directories cleanup
=========================== logs ===========================
<launching> C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe --disable-field-trial-config --disable-background-networking --enable-features=NetworkService,NetworkServiceInProcess --disable-background-timer-throttling --disable-backgrounding-occluded-windows --disable-back-forward-cache --disable-breakpad --disable-client-side-phishing-detection --disable-component-extensions-with-background-pages --disable-component-update --no-default-browser-check --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=ImprovedCookieControls,LazyFrameLoading,GlobalMediaControls,DestroyProfileOnBrowserClose,MediaRouter,DialMediaRouteProvider,AcceptCHFrame,AutoExpandDetailsElement,CertificateTransparencyComponentUpdater,AvoidUnnecessaryBeforeUnloadCheckSync,Translate --allow-pre-commit-input --disable-hang-monitor --disable-ipc-flooding-protection --disable-popup-blocking --disable-prompt-on-repost --disable-renderer-backgrounding --disable-sync --force-color-profile=srgb --metrics-recording-only --no-first-run --enable-automation --password-store=basic --use-mock-keychain --no-service-autorun --export-tagged-pdf --no-sandbox --user-data-dir=C:\Users\****\AppData\Local\Microsoft\Edge\User Data\Default --remote-debugging-pipe about:blank
<launched> pid=529300
[pid=529300][out] Opening in existing browser session.
[pid=529300] <process did exit: exitCode=0, signal=null>
[pid=529300] starting temporary directories cleanup
============================================================

解决方案 #

核心问题是多个脚本共用同一Edge用户数据目录，浏览器不允许同时从同一用户目录启动多个实例。解决思路是给每个脚本分配独立的用户数据子目录，同时保留原认证信息。

具体实现步骤 #

1. 给每个脚本设置唯一标识后缀，用于区分用户目录
2. 复制原Edge默认用户目录到独立子目录（首次运行时执行）
3. 修改脚本中的context_location为独立子目录路径

修改后的代码示例 #

import os
import shutil
from playwright.sync_api import sync_playwright

def get_cookies(context):
    global cookies
    cookie_set = {tempdict['name'] for tempdict in cookies}
    new_cookies = context.cookies()
    context_cookie_set = {tempdict['name'] for tempdict in new_cookies}
    new_cookie_set = context_cookie_set - cookie_set
    if len(new_cookie_set) == 0:
        return 0

    cookies = new_cookies
    return len(new_cookie_set)

# 每个脚本设置唯一后缀，比如脚本1用"_script1"，脚本2用"_script2"
UNIQUE_SCRIPT_SUFFIX = "_script1"
username = os.environ.get('USERNAME')
edge_user_data_root = rf'C:\Users\{username}\AppData\Local\Microsoft\Edge\User Data'
original_default_dir = os.path.join(edge_user_data_root, "Default")
# 生成当前脚本专属的用户目录
script_specific_dir = os.path.join(edge_user_data_root, f"Default{UNIQUE_SCRIPT_SUFFIX}")

# 首次运行时复制原认证目录内容
if not os.path.exists(script_specific_dir):
    shutil.copytree(original_default_dir, script_specific_dir)

cookies = []
playwright = sync_playwright().start()
context = playwright.chromium.launch_persistent_context(
    script_specific_dir,
    headless=False,
    channel='msedge'
)
get_cookies(context)
context.on('requestfinished', lambda r: get_cookies(context))
context.on('requestfailed', lambda r: get_cookies(context))
page = context.new_page()
page.goto('https://playwright.dev/python/')
page.wait_for_load_state(state='networkidle')
while get_cookies(context) > 0:
    print('.')
    pass
seconds = 30
page.wait_for_timeout(seconds * 1000)

# 运行结束后关闭上下文和Playwright实例
context.close()
playwright.stop()

注意事项 #

• 每个脚本必须使用唯一的UNIQUE_SCRIPT_SUFFIX，避免目录冲突
• 如果原用户目录的认证信息更新，需要重新复制子目录，或者手动同步关键文件（如Cookies、Login Data）
• 该方案完全基于同步API实现，无需修改原有代码结构，保持简洁性

内容的提问来源于stack exchange，提问作者Adrian