多仓库共用Python公共模块：采用子模块还是列为依赖？

Hey there! Let’s break this down based on your research team’s workflow and priorities—since you’re in a research setting (not a formal dev environment), ease of use for collaborators and reproducibility are likely your biggest concerns. Here’s a comparison of the two approaches, plus recommendations tailored to your context:

Option 1: Separate repository as a dependency #

This means turning common_analysis_tools into its own standalone repo, and listing it as a dependency in each project’s requirements.txt or pyproject.toml.

Pros: #

• Centralized maintenance: Fix a bug or add a feature once, and all projects can pull the updated version—no need to update submodules across multiple repos.
• Reproducibility clarity: You can tag versions (e.g., v1.1.0, v2.0.0) so collaborators (and anyone trying to replicate your work) can install an exact, documented version of the tools. This is huge for research reproducibility.
• Clean project structure: Each project repo only contains code specific to that research project, not shared utility code.
• Simplified installation: Collaborators can install the tools with a single command like pip install git+https://your-tools-repo.git@v1.1.0 (or even publish it to PyPI if you want to make it even easier).

Cons: #

• Minor learning curve: If your collaborators aren’t familiar with Python dependency management, you’ll need to add clear instructions in each project’s README (but this is usually just a note to run pip install -r requirements.txt).
• Version release overhead: You’ll need to tag new versions when you update the tools, which adds a small step compared to just committing changes to a submodule.

Option 2: Git Submodule #

This embeds the common_analysis_tools repo directly inside each project repo as a subdirectory.

Pros: #

• Zero dependency setup: Collaborators can clone the project with git clone --recursive (or run git submodule update --init after cloning) and have the tools ready to use immediately—no need to install separate packages. This is great for team members who aren’t comfortable with Python’s package ecosystem.
• Project-specific version locking: Each project can pin to a specific commit of the tools, so updating the tools for one project won’t accidentally break another older project.
• Visibility: The tools code is right there in the project repo, making it easy for collaborators to inspect or modify (though modifying submodules requires a bit of Git knowledge).

Cons: #

• Maintenance hassle: Every time you update the tools, you’ll need to commit the submodule reference change in every project repo that uses it. This gets tedious if you have many projects.
• Git confusion for beginners: Collaborators might accidentally commit changes to the submodule (instead of the main project) or forget to update the submodule, leading to version mismatches and broken code.
• Repo clutter: Each project repo will include the tools code as a subdirectory, which can make repo navigation slightly messier.

Recommendations for your research environment #

• Go with a separate dependency repo if:

  • You update the tools frequently and want all projects to benefit from changes quickly.
  • Reproducibility and clear versioning are critical (e.g., you’re preparing papers or shared datasets).
  • Your team is comfortable with basic Python dependency management.

• Stick with submodules if:

  • Your collaborators are less familiar with Git/Python packaging, and you want to minimize setup steps.
  • Each project needs to stay locked to a specific version of the tools to avoid breaking existing analyses.

Bonus middle ground #

If you want the best of both worlds, you can maintain a separate tools repo and include it as a subdirectory in projects (not a Git submodule) using Git subtree. This lets you pull updates from the central repo but keeps the tools code in the project for visibility—though this adds a bit more complexity than the two main options.

内容的提问来源于stack exchange，提问作者trillian