多Python模块依赖管理及IntelliJ IDEA导入报错问题咨询

First, let's address the IntelliJ IDEA error you're seeing—this is almost always an IDE configuration issue, not a problem with your dependency setup (since it works in the terminal). Here's how to fix it:

• Verify the Python interpreter: Go to Settings > Project: [Your Project Name] > Python Interpreter and make sure you've selected the correct virtual environment for the module you're working on (e.g., data-reader/venv/bin/python on Linux/macOS, data-reader/venv/Scripts/python.exe on Windows).
• Mark commons as a Sources Root: Right-click the top-level commons folder (the one containing the commons subdirectory and setup.py) in IDEA's project pane, then select Mark Directory as > Sources Root. This tells IDEA to include this directory in its Python path resolution.
• Invalidate IDE caches: Sometimes IDEA gets stuck with old cache data. Go to File > Invalidate Caches... and choose Invalidate and Restart—this will force IDEA to re-scan your project structure and dependencies.

Now, let's tackle your two core questions:

1. Recommended Dependency Addition & Management Approach #

For your multi-module setup, here's the step-by-step best practice:

• Fix relative paths in requirements.txt: Since data-reader, data-writer, and commons are sibling directories at the project root, your requirements.txt entries should use the correct relative path. Replace -e commons with -e ../commons in both data-reader/requirements.txt and data-writer/requirements.txt. This ensures pip can find the commons module when installing from each module's virtual environment.
• Standardize virtual environment activation: For each module, activate its virtual environment first, then run pip install -r requirements.txt from the module's root directory (e.g., cd data-reader && source venv/bin/activate && pip install -r requirements.txt). This guarantees the editable dependency is installed into the correct environment.
• Keep commons' build config up to date: Ensure commons has a proper build configuration. If you're still using setup.py, consider migrating to pyproject.toml (per PEP 621) for cleaner, modern setup. A minimal example for commons/pyproject.toml:

  [build-system]
  requires = ["setuptools>=61.0"]
  build-backend = "setuptools.build_meta"
  
  [project]
  name = "commons"
  version = "0.1.0"
  packages = ["commons"]
  

2. Is Editable Dependency Installation Reasonable? What Alternatives Exist? #

Editable installs (-e) are absolutely reasonable #

This is the standard approach for developing multi-module Python projects where you need to iterate on shared code (like commons) without re-installing it every time you make a change. It works perfectly for your use case, since changes to commons/utils.py will be immediately available in data-reader and data-writer without any extra steps.

Alternative approaches to consider #

• Use a single virtual environment: If your modules don't have conflicting dependency versions, you could create a single virtual environment at the project root instead of per-module environments. This simplifies IDE configuration (you only need to set one interpreter) and avoids redundant installs of commons. You'd install commons as editable once, and all modules use the same environment.
• Modern dependency managers (Poetry/Pipenv): Tools like Poetry handle virtual environments, dependency resolution, and local editable dependencies seamlessly. For example, in data-reader's pyproject.toml, you'd add:

  [tool.poetry.dependencies]
  python = "^3.8"
  commons = { path = "../commons", develop = true }
  

  Poetry automatically creates and manages the virtual environment, and IDEA has excellent built-in support for Poetry projects, reducing configuration headaches.
• Monorepo with setuptools' find_packages: If all modules are part of a single logical project, you could structure it as a monorepo with a root pyproject.toml that includes all submodules. This lets you install the entire project in editable mode, making cross-module imports work out of the box in IDEs.

内容的提问来源于stack exchange，提问作者Mousa