Scrapy与Django集成报错求助：ModuleNotFoundError模块找不到

Let's break down and fix this error step by step—this is a common path-related issue when integrating Scrapy with Django, so don't stress!

1. Correct the Import Statement in example.py #

The immediate cause of your ModuleNotFoundError is a redundant import path. In a standard Scrapy project structure, items.py lives directly under the top-level example_bot directory (the same folder that holds settings.py and the spiders subfolder).

Change line 2 in your example.py spider file from:

from example_bot.example_bot.items import ExampleDotcdComItem

To:

from example_bot.items import ExampleDotcdComItem

2. Verify Your Project Structure #

Module resolution depends heavily on correct folder organization. Your project should look like this:

example_project/  # Django project root
├── example_project/
│   ├── settings.py
│   ├── urls.py
│   └── ...
├── app/
│   ├── models.py
│   ├── admin.py
│   └── ...
└── example_bot/  # Scrapy project root
    ├── example_bot/
    │   ├── items.py
    │   ├── settings.py
    │   └── spiders/
    │       └── example.py
    └── scrapy.cfg

If your Scrapy project (example_bot) is not nested inside your Django project (example_project), add its root path to sys.path in your Scrapy settings.py to help Python locate it:

# Add this to your Scrapy settings.py
SCRAPY_PROJECT_PATH = 'C:/Users/Muhammad Shahzad/PycharmProjects/example_bot'
sys.path.insert(0, SCRAPY_PROJECT_PATH)

3. Register Your Django App #

For Scrapy to access your ExampleDotCom model, the app containing this model must be listed in INSTALLED_APPS in your Django project's settings.py:

# In example_project/example_project/settings.py
INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'app',  # Add your app name here
]

4. Validate the Model Import Path #

Double-check that the import in items.py points to the correct location. If your Django app is named app and ExampleDotCom lives in app/models.py, this line is correct:

from example_project.app.models import ExampleDotCom

Adjust the path if your app has a different name.

5. Test Imports Manually #

To confirm all paths are set up correctly, open a terminal in your Scrapy project root (the folder with scrapy.cfg) and run:

python

Then try these imports in the Python shell:

from example_bot.items import ExampleDotcdComItem
from example_project.app.models import ExampleDotCom

If these run without errors, your path issues are fixed.

Final Step: Run the Crawler #

Make sure you're in the Scrapy project root directory when executing the crawl command:

scrapy crawl example

内容的提问来源于stack exchange，提问作者shahzad ch