多级包、文件路径及子包/模块间导入技术咨询

Hey there! Let's walk through how to handle imports and file path operations in your multi-level Python package structure—this is a common scenario, so I'll break it down clearly for you.

First, let's cover how to properly import modules across different levels of your package:

From the External Script (test.py) #

Since test.py sits at the same level as main_package, you can use absolute imports directly:

# Import top-level modules in main_package
from main_package import common1, common2

# Import modules from sub-packages
from main_package.sub_package1 import lib1
from main_package.sub_package2 import lib2

# Even import specific functions/classes if needed
from main_package.sub_package1.lib1 import your_function_name

Inside the Package (Intra-Package Imports) #

For imports within the package, you have two reliable options:

Absolute Imports (Recommended for Clarity) #

Use the full package path regardless of where you're importing from. For example, in sub_package1/lib1.py:

# Import a top-level module from main_package
from main_package import common1

# Import a module from another sub-package
from main_package.sub_package2 import lib2

Relative Imports (Useful for Local Package Structure) #

If you want to reference modules relative to the current file's position, use dot notation:

# In sub_package1/lib1.py: import from the parent package (main_package)
from .. import common1

# In main_package/common1.py: import from a sub-package
from .sub_package1 import lib1

⚠️ Note: Relative imports only work when the file is treated as part of a package. If you run lib1.py directly (e.g., python main_package/sub_package1/lib1.py), Python will throw an error. Instead, run it as a module with:

python -m main_package.sub_package1.lib1

Using init.py to Simplify Imports #

Right now your __init__.py files are empty, but you can use them to expose specific components from sub-packages, making imports cleaner for external users. For example, in main_package/sub_package1/__init__.py:

# Expose functions/classes from lib1 directly under the sub_package1 namespace
from .lib1 import table_processor, load_table

Now external scripts can import directly from the sub-package without specifying the module:

from main_package.sub_package1 import table_processor

Hardcoding relative paths (like ../tables/table1.json) is risky because it depends on the working directory where you run your script. Instead, use the __file__ attribute (or pathlib for cleaner code) to build absolute paths dynamically.

Reading JSON Files in the Same Directory #

For example, reading master_table.json from main_package/common1.py:

Using os module #

import os
import json

# Get the absolute path of the current module's directory
current_dir = os.path.dirname(os.path.abspath(__file__))
# Build the path to master_table.json
master_table_path = os.path.join(current_dir, "master_table.json")

with open(master_table_path, "r") as f:
    master_table = json.load(f)

Using pathlib (More Modern & Readable) #

I recommend using Python's pathlib module for path operations—it's more intuitive and less error-prone:

from pathlib import Path
import json

# Get the parent directory of the current module
current_dir = Path(__file__).parent
master_table_path = current_dir / "master_table.json"

with open(master_table_path, "r") as f:
    master_table = json.load(f)

Reading JSON Files in Subdirectories (e.g., tables/) #

In sub_package1/lib1.py, to read table1.json:

from pathlib import Path
import json

current_dir = Path(__file__).parent
tables_dir = current_dir / "tables"
table1_path = tables_dir / "table1.json"

with open(table1_path, "r") as f:
    table1_data = json.load(f)

Resolving Paths from master_table.json #

Your master_table.json uses relative paths like sub_package1/tables—here's how to safely resolve those to absolute paths from within main_package:

from pathlib import Path
import json

# From main_package/common1.py
current_dir = Path(__file__).parent
master_table_path = current_dir / "master_table.json"

with open(master_table_path, "r") as f:
    master_table = json.load(f)

# Resolve group1's base path
group1_base = master_table["group1"]["base"]
group1_tables_dir = current_dir / group1_base

# Load all files in group1
for file_name in master_table["group1"]["files"]:
    file_path = group1_tables_dir / file_name
    with open(file_path, "r") as f:
        table_data = json.load(f)
        # Process your table data here

• Ensure the Package is in sys.path: If you run scripts from a directory other than where main_package lives, you may need to add the package's parent directory to sys.path, or install the package in editable mode with pip install -e . (requires a setup.py or pyproject.toml file).
• Avoid Running Modules Directly: As mentioned earlier, use python -m package.module instead of python package/module.py to preserve the package structure for relative imports.
• Keep init.py Minimal (Unless Needed): Empty __init__.py files are perfectly fine—only add content if you need to expose components or run package-level initialization code.

内容的提问来源于stack exchange，提问作者Rick Manix