Hydra中配置路径为父文件夹时Schema验证失效问题
问题描述
项目结构:
configs/ ├── default.yaml └── trainings ├── data_config │ └── default.yaml ├── simple.yaml └── schema.yaml
各文件内容:
app.py
from dataclasses import dataclass from enum import Enum from pathlib import Path from omegaconf import MISSING, DictConfig, OmegaConf import hydra from hydra.core.config_store import ConfigStore CONFIGS_DIR_PATH = Path(__file__).parent / "configs" TRAININGS_DIR_PATH = CONFIGS_DIR_PATH / "trainings" class Sampling(Enum): UPSAMPLING = 1 DOWNSAMPLING = 2 @dataclass class DataConfig: sampling: Sampling = MISSING @dataclass class TrainerConfig: project_name: str = MISSING data_config: DataConfig = MISSING # @hydra.main(version_base="1.2", config_path=CONFIGS_DIR_PATH, config_name="default") @hydra.main(version_base="1.2", config_path=TRAININGS_DIR_PATH, config_name="simple") def run(configuration: DictConfig): sampling = OmegaConf.to_container(cfg=configuration, resolve=True)["data_config"]["sampling"] print(f"{sampling} Type: {type(sampling)}") def register_schemas(): config_store = ConfigStore.instance() config_store.store(name="base_schema", node=TrainerConfig) if __name__ == "__main__": register_schemas() run()
configs/default.yaml
defaults: - /trainings@: simple - _self_ project_name: test
configs/trainings/simple.yaml
defaults: - base_schema - data_config: default - _self_ project_name: test
configs/trainings/data_config/default.yaml
defaults: - _self_ sampling: DOWNSAMPLING
当前用@hydra.main(version_base="1.2", config_path=TRAININGS_DIR_PATH, config_name="simple")运行时,DOWNSAMPLING能正确解析为Enum类型,结果正常。但切换到@hydra.main(version_base="1.2", config_path=CONFIGS_DIR_PATH, config_name="default")时,出现报错:
In 'trainings/simple': Could not load 'trainings/base_schema'. Config search path: provider=hydra, path=pkg://hydra.conf provider=main, path=file:///data/code/demos/hydra/configs provider=schema, path=structured:// Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
原因分析
问题出在Hydra对配置引用的解析逻辑:
- 从
trainings目录启动时,simple.yaml里的base_schema是相对引用,Hydra会优先在**结构化配置(ConfigStore)**里查找这个名称,刚好匹配到你注册的TrainerConfig,所以正常工作。 - 从
configs目录启动时,Hydra解析simple.yaml里的base_schema时,会默认把它当成相对于当前配置根目录的文件路径,也就是尝试去configs/trainings/base_schema.yaml找对应的文件,但这个文件不存在——因为base_schema是代码注册的结构化配置,不是yaml文件,所以报错。
解决办法
有两种简单修复方式:
方式1:用绝对引用指定结构化配置
修改configs/trainings/simple.yaml里的base_schema引用,加上~前缀,明确告诉Hydra这是全局注册的结构化配置,而非文件路径:
defaults: - ~base_schema # 加~前缀表示绝对引用结构化配置 - data_config: default - _self_ project_name: test
方式2:注册schema时指定group
在register_schemas函数里,给base_schema指定group为trainings,匹配配置目录结构:
def register_schemas(): config_store = ConfigStore.instance() # 指定group为trainings,对应配置目录的结构 config_store.store(name="base_schema", node=TrainerConfig, group="trainings")
这种方式下,simple.yaml里的base_schema引用无需修改,Hydra会自动在trainings分组的结构化配置中找到它。
两种方式都能解决问题,推荐方式1,更直观明确,避免和文件路径混淆。
内容的提问来源于stack exchange,提问作者Gros Lalo
相关产品推荐
相关产品推荐

