Scrapy报错KeyError: 'Spider not found: charnley' 求助

我正在编写第一个Scrapy爬虫，执行命令scrapy crawl charnley时出现报错，爬虫命名为charnley且确认无名称冲突，但仍提示找不到该爬虫。

报错信息 #

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Python310\lib\runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None,
File "C:\Python310\lib\runpy.py", line 86, in _run_code exec(code, run_globals)
File "C:\Users\shark\VisualStudioCodeProjects\env\Scripts\scrapy.exe\__main__.py", line 
7, in <module>
File "C:\Users\shark\VisualStudioCodeProjects\env\lib\site-packages\scrapy\cmdline.py", 
line 154, in execute
_run_print_help(parser, _run_command, cmd, args, opts)
File "C:\Users\shark\VisualStudioCodeProjects\env\lib\site-packages\scrapy\cmdline.py", 
line 109, in _run_print_help
func(*a, **kw)
File "C:\Users\shark\VisualStudioCodeProjects\env\lib\site-packages\scrapy\cmdline.py", 
line 162, in _run_command
cmd.run(args, opts)
File "C:\Users\shark\VisualStudioCodeProjects\env\lib\site- 
packages\scrapy\commands\crawl.py", line 22, in run
crawl_defer = self.crawler_process.crawl(spname, **opts.spargs)
File "C:\Users\shark\VisualStudioCodeProjects\env\lib\site-packages\scrapy\crawler.py", 
line 204, in crawl
crawler = self.create_crawler(crawler_or_spidercls)
File "C:\Users\shark\VisualStudioCodeProjects\env\lib\site-packages\scrapy\crawler.py", 
line 237, in create_crawler
return self._create_crawler(crawler_or_spidercls)
File "C:\Users\shark\VisualStudioCodeProjects\env\lib\site-packages\scrapy\crawler.py", 
line 312, in _create_crawler
spidercls = self.spider_loader.load(spidercls)
File "C:\Users\shark\VisualStudioCodeProjects\env\lib\site- 
packages\scrapy\spiderloader.py", line 77, in load
raise KeyError(f"Spider not found: {spider_name}")
KeyError: 'Spider not found: charnley'

爬虫代码 #

from gc import callbacks
import scrapy

class CharnleySpider(scrapy.Spider):
    name = 'charnley'
    start_urls = ['https://www.charnleys.co.uk/shop/bentley-supreme-apex/']

    def parse(self, response):
        for elements in response.css('#tab-description'):
            yield {
                'name': elements.css('h3::text').get(),
                'price': elements.css('div:nth-child(3)').get().replace('\n',''),    
            }

问题排查与解决 #

出现Spider not found错误，核心是Scrapy无法识别你的爬虫类，按以下步骤排查：

• 检查文件位置：爬虫代码必须放在Scrapy项目的spiders目录下，确认.py文件路径正确。
• 验证配置项：打开项目的settings.py，检查SPIDER_MODULES配置是否为['你的项目名.spiders']，确保路径包含爬虫模块。
• 确认命名规范：爬虫类必须继承scrapy.Spider，且name属性拼写完全正确（大小写敏感）。
• 核对虚拟环境：执行命令前确保激活了项目对应的虚拟环境，避免使用全局Scrapy导致无法找到项目内的爬虫。
• 检查命令路径：必须在包含scrapy.cfg文件的项目根目录下执行scrapy crawl charnley。

另外，代码中from gc import callbacks是多余导入，可删除以避免潜在问题。

内容的提问来源于stack exchange，提问作者Bastion