带子模块的Python模块打包最佳实践及问题咨询

Hey there! Let's break down your module structuring questions, with a focus on aligning with numpy/scipy's conventions since that's your priority.

1. How to prevent submodules from exposing cr and np? #

The key here is to enforce clear boundaries between internal implementation details and public API—exactly how numpy/scipy structures their modules. Here are two actionable fixes:

• Mark internal dependencies as private
  In your core_sub_one.py and core_sub_two.py, rename imports with an underscore prefix to signal they're internal (a standard Python convention widely used in numpy):

  # core_sub_one.py
  import numpy as _np  # Private internal dependency
  from .. import core as _cr  # Private internal dependency
  
  def some_function_in_sub_one():
      return _np.array([1,2,3])  # Use the private alias internally
  
  # Explicitly define public API
  __all__ = ['some_function_in_sub_one']
  

  Underscore-prefixed names won't be included when users run from mdl.so import *, and they'll signal to users these aren't part of the supported public API.

• Clean up submodule __init__.py files
  Instead of importing the entire internal module and using * (which pulls in all names, including private ones), explicitly import only the public API you want to expose. For example, in sub_one/__init__.py:

  # sub_one/__init__.py
  from .core_sub_one import some_function_in_sub_one
  
  # Define the public API for the submodule
  __all__ = ['some_function_in_sub_one']
  

  This way, the sub_one namespace only contains your intended public functions—no cr, np, or core_sub_one module object cluttering things up.

If you want to mirror numpy's exact approach (like numpy.ma), they do import internal modules in submodule __init__.py files but rely on __all__ to hide internal names from the public API. The underscore prefix adds an extra layer of clarity for users.

2. What are the downsides if this issue isn't fixed? #

• API instability: Users might accidentally rely on internal names like mdl.so.np or mdl.so.cr in their code. If you ever refactor internal dependencies (e.g., switch to a different numerical library, or restructure the core module), their code will break unexpectedly.
• Namespace clutter: When users explore your module (via dir(mdl.so) or auto-completion), they'll see irrelevant internal names mixed in with your public API. This makes your module harder to learn and use.
• Violation of encapsulation: Good module design follows the "least exposure" principle—only show users what they need to use, and hide how you implement it. Exposing internal dependencies breaks this, making your module harder to maintain over time.

3. Best practices for importing shared dependencies like numpy across submodules (numpy/scipy style) #

Numpy and scipy follow these core principles for handling shared dependencies:

• Each submodule imports dependencies independently: Almost every numpy submodule starts with import numpy as np—they don't rely on importing numpy from a parent module. This keeps submodules self-contained and easier to refactor or reuse later.
• Private aliases for internal use: As mentioned earlier, using underscore-prefixed aliases (like _np) for dependencies makes it clear they're not part of the public API.
• Explicit __all__ for every module: Every public module (and submodule) defines an __all__ list that explicitly lists all public API members. This controls what gets imported with from module import * and serves as built-in documentation for users.
• Minimize top-level imports: Avoid importing entire submodules or internal files into the top-level namespace unless they're part of the intended public API. For example, numpy exposes some core functions directly in numpy instead of forcing users to dig into numpy.core, but this is a deliberate usability choice, not a default.

内容的提问来源于stack exchange，提问作者JE_Muc