Windows系统下Ray Tune fit()函数文件找不到错误求助

问题场景 #

在Windows环境中使用Ray Tune进行LightGBM模型超参数调优，代码如下：

def ray_tune_lgbm_train(config):

    X_train = ctgan_syn_X_train.copy()
    y_train = y_train.copy()
    
    opt_X_train, opt_X_val, opt_y_train, opt_y_val = train_test_split(x_train, y_train, test_size=0.2)
    train_set = lgb.Dataset(opt_X_train, label=opt_y_train)
    valid_set = lgb.Dataset(opt_X_val, label=opt_y_val)
    
    lgbm_model = lgb.train(
        config,
        train_set,
        valid_sets=[valid_set],
        valid_names=['lgbm_valid'],
        verbose_eval=False,
        callbacks=[
            TuneReportCheckpointCallback(
                {
                    'average_precision': 'val_avp',
                    'auc': 'val_auc',
                }
            )
        ],
    )
    y_val_pred = lgbm_model.predict(opt_X_val)
    session.report(
        {
            "AUC": roc_auc_score(opt_y_val, y_val_pred),
            "done": True,
        }
    )
    
    
config = {
    "objective": "binary",
    "metric": "auc",
    "random_state": 10,
    "verbosity": -1,
    "boosting": "gbdt",
    "num_threads": 4,

    "num_leaves": tune.randint(4, 30),
    "learning_rate": tune.loguniform(0.005, 1.0),
    "bagging_fraction": tune.uniform(0.1, 1.0),
    "feature_fraction": tune.uniform(0.1, 1.0),
    "bagging_freq": tune.randint(10, 30),
    "min_data_in_leaf": tune.randint(1000, 3000),
    "num_iterations": tune.randint(1000, 3000)
}

tuner = tune.Tuner(
    ray_tune_lgbm_train,
    tune_config=tune.TuneConfig(
        metric='auc',
        mode="max",
        scheduler=ASHAScheduler(),
        num_samples=5,
    ),
    param_space=config,
)

results = tuner.fit()

运行时出现以下错误：

File ~\anaconda3\envs\secret_guest\lib\site-packages\tensorboardX\record_writer.py:58, in open_file(path)
57 prefix = path.split(':')[0]
---> 58 factory = REGISTERED_FACTORIES[prefix]
59 return factory.open(path)

KeyError: 'C'

FileNotFoundError: [Errno 2] No such file or directory: 'C:\ray_results\ray_tune_lgbm_train_2023-09-13_23-23-51\ray_tune_lgbm_train_23db6_00002_2_bagging_fraction=0.7080,bagging_freq=15,feature_fraction=0.4676,learning_rate=0.1459,min_data_in_2023-09-13_23-23-58\events.out.tfevents.1694661844.DESKTOP-BB1U7JL'

可找到对应输出文件夹，但找不到文件events.out.tfevents.1694661844.DESKTOP-BB1U7JL。

解决方法 #

1. 修改Ray输出路径为相对路径 #

错误根源是tensorboardX误将Windows盘符（如C:）解析为URI前缀，改用相对路径可规避该问题：

from ray.tune import RunConfig

tuner = tune.Tuner(
    ray_tune_lgbm_train,
    tune_config=tune.TuneConfig(
        metric='auc',
        mode="max",
        scheduler=ASHAScheduler(),
        num_samples=5,
    ),
    param_space=config,
    run_config=RunConfig(
        storage_path="./ray_results",  # 用相对路径替代绝对路径
        name=None  # 让Ray自动生成文件夹名，减少路径长度
    )
)

2. 禁用TensorBoard日志 #

若不需要TensorBoard监控，直接关闭日志输出可从根源避免路径问题：

run_config=RunConfig(
    log_to_file=False,
    callbacks=[tune.logger.NoOpLoggerCallback()]
)

3. 升级tensorboardX或替换日志组件 #

旧版本tensorboardX对Windows路径支持存在缺陷，升级到最新版：

pip install --upgrade tensorboardX

或改用Ray官方兼容的TensorBoard日志组件：

from ray.tune.logger import TBXLogger

tuner = tune.Tuner(
    ray_tune_lgbm_train,
    tune_config=tune.TuneConfig(...),
    param_space=config,
    run_config=RunConfig(
        callbacks=[TBXLogger()]
    )
)

4. 解决Windows长路径限制 #

Windows默认文件路径最大长度为260字符，当前路径已接近阈值，可通过以下方式解决：

• 缩短storage_path层级，比如改用./ray_out作为输出目录
• 启用Windows长路径支持：
  1. 打开注册表编辑器，定位到HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem
  2. 将LongPathsEnabled的值修改为1（不存在则新建DWORD值）
  3. 重启电脑生效

内容的提问来源于stack exchange，提问作者Cherry Wu