如何配置Python项目使pytest与Jupyter可导入src目录模块？

问题场景 #

项目目录结构如下：

project/
├── Dockerfile
├── requirements.txt
├── pyproject.toml
├── jupyter_notebook_config.py
├── notebooks/
│   ├── src/
│   │   ├── my_module.py
│   │   └── module_name.c
│   ├── tests/
│   │   ├── test_my_module.py
│   │   └── test_sub_module.py
│   └── my_notebook.ipynb
└── ...

已通过pytest-pythonpath插件在pyproject.toml配置pytest路径：

# pyproject.toml
[tool.pytest.ini_options]
python_paths = ["src"]

Docker基于python:3.11镜像构建，配置如下：

FROM python:3.11

# Copy the requirements.txt file to the container
COPY requirements.txt /app/requirements.txt

# Install the dependencies from the requirements.txt file
RUN pip install -r /app/requirements.txt

# Copy the pyproject.toml file to the container
COPY pyproject.toml /app/pyproject.toml

# Set working directory
WORKDIR /app

# Run pip install -e . to install pyproject.toml
RUN pip install -e .

# Expose Jupyter Notebook port
EXPOSE 8888

# Copy the custom Jupyter configuration file
COPY jupyter_notebook_config.py /root/.jupyter/

# Run Jupyter Notebook on container startup
CMD ["jupyter", "notebook", "--ip=0.0.0.0", "--port=8888", "--no-browser", "--allow-root"]

尝试在测试文件和Notebook开头手动添加路径：

import os
import sys

sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), 'notebooks/src')))

但pytest仍报错：

============================= test session starts ==============================
platform linux -- Python 3.11.4, pytest-6.2.5, py-1.11.0, pluggy-1.2.0
rootdir: /app
plugins: pythonpath-0.7.4, anyio-3.7.1
collected 0 items / 1 error                                                    

==================================== ERRORS ====================================
_______________ ERROR collecting tests/test_my_functions_demo.py _______________
ImportError while importing test module '/app/tests/test_my_functions_demo.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/local/lib/python3.11/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_my_functions_demo.py:2: in <module>
    from my_functions_demo import add, subtract
E   ModuleNotFoundError: No module named 'my_functions_demo'
=========================== short test summary info ============================
ERROR tests/test_my_functions_demo.py
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 0.15s ===============================

目前仅能通过import src.module_name导入模块，直接import module_name失败，需要实现pytest和Jupyter都能直接访问src下的模块。

解决步骤 #

1. 调整项目结构 #

把notebooks/src目录移动到项目根目录，新结构如下：

project/
├── src/
│   ├── my_module.py
│   └── module_name.c
├── notebooks/
│   ├── tests/
│   │   ├── test_my_module.py
│   │   └── test_sub_module.py
│   └── my_notebook.ipynb
├── Dockerfile
├── requirements.txt
├── pyproject.toml
├── jupyter_notebook_config.py
└── ...

2. 完善pyproject.toml配置 #

添加项目包声明，让pip install -e .能识别src目录为可导入包：

[project]
name = "your-project-name" # 替换为你的项目名
version = "0.1.0"
packages = ["src"]

[tool.pytest.ini_options]
python_paths = ["src"]

3. 修正Dockerfile复制逻辑 #

原来的Dockerfile没复制src和notebooks目录，修改后确保所有必要文件都被复制：

FROM python:3.11

# 先设置工作目录
WORKDIR /app

# 复制所有需要的文件和目录
COPY requirements.txt .
COPY pyproject.toml .
COPY src/ ./src/
COPY notebooks/ ./notebooks/
COPY jupyter_notebook_config.py /root/.jupyter/

# 安装依赖和可编辑包
RUN pip install -r requirements.txt
RUN pip install -e .

# 暴露Jupyter端口
EXPOSE 8888

# 启动Jupyter Notebook
CMD ["jupyter", "notebook", "--ip=0.0.0.0", "--port=8888", "--no-browser", "--allow-root"]

4. 清理测试文件的手动路径代码 #

删除测试文件里添加sys.path的代码，直接使用from my_module import ...导入模块即可，因为pip install -e .已经把src加入Python全局路径，pytest的python_paths配置也会生效。

5. Jupyter Notebook配置 #

不需要在Notebook里手动加路径，直接import my_module即可。如果需要实时重载修改后的模块，可以在Notebook开头执行：

%load_ext autoreload
%autoreload 2

6. 验证配置 #

• 进入容器后，运行pytest notebooks/tests/，确认测试能正常收集和执行。
• 启动Jupyter后打开my_notebook.ipynb，测试import my_module是否成功。

内容的提问来源于stack exchange，提问作者Bosser445