Scrapy集成Playwright报错：无法导入PageCoroutine

问题原因 #

PageCoroutine类的导入路径在scrapy-playwright的新版本中发生了变更：旧版本中它位于scrapy_playwright.page模块，新版本已移至scrapy_playwright.handlers模块；此外，新版本也支持更简洁的元组写法定义页面协程，无需导入该类。

解决方案 #

方案1：修正导入路径 #

将导入语句修改为：

from scrapy_playwright.handlers import PageCoroutine

代码其余部分无需改动，修改后即可正常运行。

方案2：使用元组替代PageCoroutine（推荐） #

新版本scrapy-playwright支持直接用元组定义页面协程，无需导入额外类，代码更简洁：

import scrapy

class PwspiderSpider(scrapy.Spider):
    name = 'pwspider'
    
    def start_requests(self):
        yield scrapy.Request(
            "https://shoppable-campaign-demo.netlify.app/#/",
            meta=dict(
                playwright=True,
                playwright_include_page=True,
                playwright_page_coroutines=[('wait_for_selector', 'div#productListing')]
            )
        )

    async def parse(self, response):
        yield {'text': response.text}

注意：这里将playwright_page_coroutine改为复数形式playwright_page_coroutines，这是新版本的规范写法，单数形式虽兼容但推荐使用复数。

额外检查 #

确保你的settings.py配置正确：

DOWNLOAD_HANDLERS = {
    "http": "scrapy_playwright.handler.ScrapyPlaywrightDownloadHandler",
    "https": "scrapy_playwright.handler.ScrapyPlaywrightDownloadHandler",
}

TWISTED_REACTOR = "twisted.internet.asyncioreactor.AsyncioSelectorReactor"

内容的提问来源于stack exchange，提问作者Ali_Khaled