Watchdog库长期运行失效问题求助
我有一个使用watchdog库监控服务器文件夹的Python脚本,当有文件创建时会执行相应处理。该脚本通过nssm注册为系统服务,初始几小时运行正常,但后续虽服务仍处于运行状态却无任何响应,EventViewer及应用日志中均未发现报错或警告信息。
我的代码是基于watchdog库的简单实现,具体如下:
import time import logging import signal from watchdog.observers import Observer from watchdog.events import FileSystemEventHandler from logging.handlers import TimedRotatingFileHandler from project import Program from config import Config class MonitorFolder(FileSystemEventHandler): def __init__(self, config: Config) -> None: self.logfile_path = config.config.get('settings','logfile_path') self.monitor_path = str(config.config.get('settings','monitor_path')).replace(r'\\',r'\\') #----------------Logging--------------------------------- ##FileHandler logger=logging.getLogger() logger.setLevel(logging.INFO) handler=TimedRotatingFileHandler(filename = self.logfile_path, when ='D', interval = 1) formatter=logging.Formatter('%(levelname)s %(asctime)s [%(name)s.%(funcName)s:%(lineno)d] %(message)s') handler.setFormatter(formatter) logger.addHandler(handler) # ##ConsoleHandler console=logging.StreamHandler() console.setFormatter (formatter) console.setLevel(logging.INFO) logging.getLogger().addHandler(console) #----------------Logging--------------------------------- def on_created(self, event): if ('Archive' in event.src_path or 'Error' in event.src_path ): return logging.info(f'Folder monitor received "{event.src_path}" event for "{event.event_type}"') time.sleep(5) prog = Program(event.src_path, config) prog.main() if __name__ == "__main__": config = Config() event_handler = MonitorFolder(config) observer = Observer() observer.schedule(event_handler, path = event_handler.monitor_path, recursive = True) logging.info("Monitoring started...") observer.start() try: while(True): time.sleep(1) except Exception as e: logging.error(e) observer.stop() observer.join()
此外,应用日志文件中也未记录任何错误。请问有没有人遇到过类似问题?
事件处理线程被阻塞:Watchdog的Observer依赖事件线程处理文件变化,你在
on_created里直接调用prog.main(),如果这个方法执行耗时久、陷入死循环或者死锁,会直接阻塞事件线程,后续的文件创建事件根本无法被处理。建议把业务逻辑放到单独线程执行:import threading def on_created(self, event): if ('Archive' in event.src_path or 'Error' in event.src_path ): return logging.info(f'Folder monitor received "{event.src_path}" event for "{event.event_type}"') time.sleep(5) # 用独立线程执行处理逻辑,避免阻塞Observer的事件循环 thread = threading.Thread(target=lambda: Program(event.src_path, config).main()) thread.daemon = True thread.start()日志配置重复添加Handler:你在
MonitorFolder的__init__里给根logger重复添加Handler,每次实例化都会加一遍,日志输出时可能触发锁竞争,导致进程卡死。建议把日志配置移到主入口,只初始化一次:if __name__ == "__main__": config = Config() # 提前初始化日志,避免重复添加Handler logfile_path = config.config.get('settings','logfile_path') logger = logging.getLogger() logger.setLevel(logging.INFO) handler = TimedRotatingFileHandler(filename=logfile_path, when='D', interval=1) formatter = logging.Formatter('%(levelname)s %(asctime)s [%(name)s.%(funcName)s:%(lineno)d] %(message)s') handler.setFormatter(formatter) logger.addHandler(handler) console = logging.StreamHandler() console.setFormatter(formatter) console.setLevel(logging.INFO) logger.addHandler(console) event_handler = MonitorFolder(config) observer = Observer() observer.schedule(event_handler, path=event_handler.monitor_path, recursive=True) logging.info("Monitoring started...") # ... 后续代码不变主线程循环方式不合理:你用
while True: time.sleep(1)维持主线程,在Windows服务环境下,这种空闲循环可能被系统判定为低优先级进程,甚至触发资源回收导致Observer线程挂起。可以直接用Observer的join()方法替代:if __name__ == "__main__": # ... 日志和初始化代码 observer.start() try: observer.join() # 直接等待Observer线程结束,无需自定义sleep循环 except KeyboardInterrupt: observer.stop() observer.join()业务逻辑中的隐性问题:
prog.main()里可能存在未捕获的异常、资源泄漏(比如文件句柄、数据库连接未关闭),长时间运行后积累导致进程卡死。建议在prog.main()中添加完整的异常捕获,把错误写入日志;同时检查所有资源是否正确释放。
内容的提问来源于stack exchange,提问作者Sbeastan

