使用uvloop事件循环退出异步函数时出现Observer错误

现象 #

• 启用uvloop时，调用observer.stop()和observer.join()会触发ImportError: sys.meta_path is None, Python is likely shutting down错误
• 不使用uvloop，或者不调用这两个方法时无异常
• 无论在异步代码（选项1）还是同步代码（偏好选项2）中调用，都会出现错误
• 场景：用Observer监听配置文件，程序关闭时需要停止监听线程

错误信息 #

ImportError: sys.meta_path is None, Python is likely shutting down

相关代码 #

import sys
import os
import asyncio
import logging

if sys.platform != 'win32':
    import uvloop
    uvloop.install()


async def main(loop_container, connector, processor, observer):

    loop_container.set_current_event_loop()

    try:
        connector.connect_exchange()
        asyncio.create_task(processor.process_queue_events())
        is_connected = await connector.is_connected()
        await connector.start_symbol_watchings()

        retry_delay = 1
        while True:
            await asyncio.sleep(retry_delay)    # keepalive
            # retry logic (...)

    except asyncio.CancelledError:
        logging.error(f'{Utils.get_current_datetime()}  Execution has been cancelled.')

    except Exception as exc:
        logging.error(f'Unknown Error:  {type(exc).__name__}  {str(exc)}')

    finally:
        await connector.disconnect_exchange()

        # Option 1
        #observer.stop()
        #observer.join()

if __name__ == "__main__":

    try:

        tick_data_queue = asyncio.Queue()
        observer        = Observer()
        loop_container  = EventLoopContainer()

        config_abs_path = os.path.abspath(CONFIG_FILE_NAME)
        output_abs_path = os.path.abspath(OUTPUT_FILE_NAME)
        config_reader   = ConfigReader(config_abs_path)
        state_persister = StatePersister(output_abs_path, config_reader)

        connector       = ExchangeConnector(loop_container, tick_data_queue, config_reader, state_persister)
        processor       = TickProcessor(loop_container, tick_data_queue, config_reader, state_persister, connector)
        event_handler   = FileChangeHandler(loop_container, config_reader, state_persister, connector, processor)

        observer.schedule(
            event_handler,
            path        = os.path.dirname(config_abs_path),
            recursive   = False,
        )
        observer.start()

        main_coro = main(loop_container, connector, processor, observer)
        asyncio.run(main_coro)

        # Option 2 (preferred)
        #observer.stop()
        #observer.join()

    except Exception as exc:
        logging.critical('Terminated.')

错误日志 #

^C
2024-11-14|17:25:17.996  Execution has been cancelled.
2024-11-14|17:25:18.001  [symbol='ETH/USDT:USDT']  Connection error: ExchangeClosedByUser : Connection closed by the user.
2024-11-14|17:25:18.002  [symbol='BTC/USDT:USDT']  Connection error: ExchangeClosedByUser : Connection closed by the user.
2024-11-14|17:25:18.253  Disconnected from the exchange.
--- Logging error ---
--- Logging error ---
--- Logging error ---
--- Logging error ---
Exception ignored in: <bound method Loop.call_exception_handler of <uvloop.Loop running=False closed=True debug=False>>
Traceback (most recent call last):
  File "uvloop/loop.pyx", line 2429, in uvloop.loop.Loop.call_exception_handler
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/logging/__init__.py", line 1548, in error
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/logging/__init__.py", line 1664, in _log
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/logging/__init__.py", line 1680, in handle
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/logging/__init__.py", line 1736, in callHandlers
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/logging/__init__.py", line 1026, in handle
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/logging/handlers.py", line 83, in emit
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/logging/__init__.py", line 1075, in handleError
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/traceback.py", line 129, in print_exception
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/traceback.py", line 1044, in __init__
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/traceback.py", line 492, in _extract_from_extended_frame_gen
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/traceback.py", line 369, in line
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/traceback.py", line 350, in _set_lines
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/linecache.py", line 25, in getline
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/linecache.py", line 41, in getlines
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/linecache.py", line 88, in updatecache
ImportError: sys.meta_path is None, Python is likely shutting down
--- Logging error ---
--- Logging error ---
--- Logging error ---
--- Logging error ---
Exception ignored in: <bound method Loop.call_exception_handler of <uvloop.Loop running=False closed=True debug=False>>
Traceback (most recent call last):
  File "uvloop/loop.pyx", line 2429, in uvloop.loop.Loop.call_exception_handler
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/logging/__init__.py", line 1548, in error
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/logging/__init__.py", line 1664, in _log
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/logging/__init__.py", line 1680, in handle
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/logging/__init__.py", line 1736, in callHandlers
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/logging/__init__.py", line 1026, in handle
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/logging/handlers.py", line 83, in emit
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/logging/__init__.py", line 1075, in handleError
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/traceback.py", line 129, in print_exception
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/traceback.py", line 1044, in __init__
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/traceback.py", line 492, in _extract_from_extended_frame_gen
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/traceback.py", line 369, in line
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/traceback.py", line 350, in _set_lines
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/linecache.py", line 25, in getline
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/linecache.py", line 41, in getlines
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/linecache.py", line 88, in updatecache
ImportError: sys.meta_path is None, Python is likely shutting down

解决方案 #

这个错误的核心是Python解释器关闭阶段，uvloop事件循环的残留回调尝试访问已销毁的系统模块（sys.meta_path），Observer线程的stop/join操作刚好触发了这个时间窗口的冲突。以下是几种可行的解决方法：

方案1：提前在异步循环内停止Observer #

把Observer的stop/join移到asyncio.run()之前的异步上下文里，确保在事件循环未关闭、Python解释器未进入销毁阶段时完成线程清理：

# 修改main函数的finally块
finally:
    await connector.disconnect_exchange()
    # 启用Option1代码，在异步循环内处理线程停止
    observer.stop()
    observer.join()

同时注释掉Option2的代码，这样线程清理在事件循环运行期间完成，不会触发模块访问错误。

方案2：给join操作添加超时（适配偏好的Option2） #

如果坚持用同步方式停止Observer，给join()添加超时，避免线程清理拖到Python开始销毁系统模块：

# 修改Option2代码
observer.stop()
observer.join(timeout=2)  # 等待2秒，超时后强制退出

超时后即使线程未完全结束，Python解释器会自动终止守护线程（若Observer是守护线程），避免后续错误。

方案3：将Observer设为守护线程 #

启动Observer前设置其为守护线程，Python解释器退出时会自动终止该线程，无需手动调用stop/join：

observer = Observer()
observer.daemon = True  # 设置为守护线程
observer.start()

这种方式无需手动管理线程停止，能彻底避免该错误，适合对线程清理要求不严格的场景。

方案4：延迟uvloop安装时机 #

不在模块顶部安装uvloop，而是在if __name__ == "__main__"块内、创建事件循环前安装：

if __name__ == "__main__":
    if sys.platform != 'win32':
        import uvloop
        uvloop.install()
    # 后续代码不变

减少uvloop在模块导入阶段的残留影响，降低解释器销毁时的冲突概率。

内容的提问来源于stack exchange，提问作者elaspog