跨文件夹导入Python模块失败及类型注解问题求助

It looks like the main issues here are missing package structure setup and incorrect sys.path configuration. Let's break down the solutions step by step:

1. Make Tokenizer a Proper Python Package #

Python requires an __init__.py file in a directory to recognize it as an importable package. Add an empty __init__.py file inside your Tokenizer folder:

Parent
├── Tokenizer
│   ├── __init__.py  # Add this empty file
│   ├── BPE.py
│   └── GPT
└── DataLoader.py

This tells Python that Tokenizer is a valid package containing modules like BPE.py.

2. Correct the Import Statement #

Once you have the __init__.py file, you have two reliable ways to import BPE:

Option A: Run Your Script from the Parent Directory #

Open your terminal, navigate to the Parent folder, and run your DataLoader.py script directly:

cd /path/to/Parent
python DataLoader.py

Now you can use your original import line without any issues:

from Tokenizer.BPE import BPE

class DataLoader:
    def __init__(self, text: str, tokenizer: BPE, context_size: int):
        # Your implementation here

This works because when you run the script from Parent, the Parent directory is automatically added to Python's system path (sys.path), so it can find the Tokenizer package.

Option B: Dynamically Add Parent to sys.path (For Running from Any Directory) #

If you need to run the script from a different directory, add this code at the very top of DataLoader.py (before any other imports):

import os
import sys

# Get the absolute path of the Parent directory (where DataLoader.py resides)
parent_directory = os.path.abspath(os.path.dirname(__file__))
# Add Parent to the start of sys.path so Python can locate the Tokenizer package
sys.path.insert(0, parent_directory)

# Now import BPE normally
from Tokenizer.BPE import BPE

class DataLoader:
    def __init__(self, text: str, tokenizer: BPE, context_size: int):
        # Your implementation here

The mistake in your original sys.path attempt was adding the path to BPE.py directly—Python looks for directories (packages) in sys.path, not individual files. Adding the Parent directory fixes this.

Why This Fixes the Type Annotation Issue #

Once you can properly import the BPE class using either method above, Python will recognize BPE as a valid type, so your tokenizer: BPE annotation will work as expected. No more unresolved reference errors!

内容的提问来源于stack exchange，提问作者Bakr MARHFOUL