Search code examples
pythonfb-hydraomegaconf

Is it possible to use the override syntax with structured configs in Hydra?


The Hydra docs on configuring experiments explains how to use overrides in standard, file-base configs. Is it possible to use overrides in Structured Configs? A minimum example:

from dataclasses import dataclass
from hydra.core.config_store import ConfigStore
from omegaconf import DictConfig, MISSING

@dataclass
class MyConfig(DictConfig):
    data: DataConfig = MISSING
    model: ModelConfig = MISSING
    parameters: dict[str, Any] = MISSING

cs = ConfigStore.instance()

cs.store(name="my_conf", node=MyConfig)
cs.store(group="model", name="v1", node=ModelConfig_V1)
cs.store(group="data", name="synthetic", node=DataConfig_Synthetic)

I would like to add in experiments somewhat like this:

@dataclass
class ExperimentConfig_Default:
    description: str = "Some experiment."
    parameters: dict[str, Any] = field(default_factors=lambda:{"lr":0.01})

cs.store(group="experiment", name="default", node=ExperimentConfig_Default)

I want to be able to use the CLI like this:

python main.py model=v1 data=synthetic +experiment=default 

to get a config like this:

data:
  ...
model:
  ...
parameters:
  lr: 0.01

Solution

  • Yes, this is possible. At runtime the only difference between Structured Configs and file-based configs is that Structured Configs have runtime type checks for config composition/modification.

    When registering a config in the config store, you can also specify the package to be global:

    cs.store(
      group="experiment", 
      name="default", 
      node=ExperimentConfig_Default, 
      package="_global_"
    )