运行Scrapy+Playwright代码报错：'PipeTransport'对象无'_output'属性

这个错误大多是Playwright与Scrapy-Playwright版本不兼容或项目配置缺失导致的，以下是针对性解决步骤：

1. 锁定兼容版本 #

新版playwright-python调整了内部PipeTransport相关API，旧版scrapy-playwright未适配会触发该错误。执行以下命令安装稳定兼容的版本组合：

pip uninstall -y scrapy-playwright playwright
pip install scrapy-playwright==0.0.29 playwright==1.32.0

2. 完善Scrapy项目配置 #

确保在项目的settings.py中添加Playwright相关配置，启用下载处理器和中间件：

# 替换默认下载处理器
DOWNLOAD_HANDLERS = {
    "http": "scrapy_playwright.handler.ScrapyPlaywrightDownloadHandler",
    "https": "scrapy_playwright.handler.ScrapyPlaywrightDownloadHandler",
}

# 启用Playwright中间件
DOWNLOADER_MIDDLEWARES = {
    "scrapy.downloadermiddlewares.useragent.UserAgentMiddleware": None,
    "scrapy_playwright.middleware.PlaywrightMiddleware": 543,
}

# 配置Playwright启动参数（可选）
PLAYWRIGHT_LAUNCH_OPTIONS = {
    "headless": True,
    "timeout": 30000,
}

3. 确保浏览器驱动安装完整 #

运行以下命令安装Playwright所需的浏览器及驱动：

playwright install

4. 代码优化（可选） #

可以尝试改用scrapy_playwright.requests.PlaywrightRequest替代原生scrapy.Request，兼容性更好：

import scrapy
from scrapy_playwright.requests import PlaywrightRequest

class JobsSpider(scrapy.Spider):
    name = 'jobs'

    def start_requests(self):
        yield PlaywrightRequest(
            url='https://jobs.goodlifefitness.com/listjobs/',
            callback=self.parse
        )

    def parse(self, response):
        yield {'text': response.text}

内容的提问来源于stack exchange，提问作者codingLearner99