How to allow the user to override a subset of the configuration using their own yaml file?

Let's say I have this basic app:

from dataclasses import dataclass

import hydra
from hydra.core.config_store import ConfigStore


@dataclass
class MyAppConfig:
    req_int: int
    opt_str: str = "Default String"
    opt_float: float = 3.14


cs = ConfigStore.instance()
# Registering the Config class with the name 'config'.
cs.store(name="base_config", node=MyAppConfig)


@hydra.main(version_base=None, config_name="base_config", config_path="conf")
def my_app(cfg: MyAppConfig) -> None:
    print(cfg)


if __name__ == "__main__":
    my_app()

Is it possible for the user to be able to call my app like this:

python my_app.py req_int=42 --config="~/path/to/user-defined-config.yaml"

And user-defined-config.yaml would contain only this:

opt_str: User Config String

The output should look like this:

{'req_int': 42, 'opt_str': 'User Config String', 'opt_float': 3.14, 'config': 'hydra-user-conf'}

The closest I got to that is: user-defined-config.yaml

defaults:
  - base_config
  - _self_

opt_str: User Config String

And the invocation:

python hydra/app.py req_int=42 --config-path='~/path/to' --config-name="hydra-user-conf"

But this way the user (who I don't want to require to be familiar with hydra) has to specify the path to their config file via two cli arguments and also include the defaults section in their config, which would be redundant boilerplate to them if they have to always include it in all of their configuration files.

Is this the closest I can get with hydra to the desired interface?

Solution

One thing you can do is to pre-configure the config searchpath in the primary config. Adding something like ~/.my_app/ to your config searchpath (thus potentially eliminating the need for --config-path|-cp.

In yaml it would look like:

hydra:
  searchpath:
    - file://${oc.env:HOME}/.my_app

Another thing to consider is having the app generating an initial config for the user on demand. I took this approach with Configen.

In general, the current patterns are not amazing and maybe there is room for some improvements in Hydra to make this more ergonomic (You can open a discussion about it).