Selecting multiple configs from a Config Group in Hydra without using an explicit nesting level inside each of the configs

Question

The documentation in https://hydra.cc/docs/patterns/select_multiple_configs_from_config_group/ shows how to pick multiple configs from a Config Group and place them in a dictionary-like structure.

However, as mentioned in the very last paragraph there, "example uses an explicit nesting level inside each of the configs to prevent them stepping over one another". For my use-case, this would prove extremely cumbersome and I would like to avoid it at all costs if possible.

Is there a way to achieve a similar result without resorting to explicitly adding the level in the individual configs? Thanks in advance :)

Answer 1

You can use a defaults-list @package keyword to achieve a similar result.

Let's assume you have changed the yaml files server/site/fb.yaml and server/site/google.yaml to not contain the "explicit nesting", so e.g. server/site/fb.yaml contains only the data domain: facebook.com.

You can achieve the same output as from the docs webpage using the following defaults list in server/apache.yaml`:

# Option 1: With a non-overridable defaults list
defaults:
  - site/fb@site.fb
  - site/google@site.google
# Option 2: With an overridable defaults list
defaults:
  - site@site.fb: fb
  - site@site.google: google

Either option 1 or option 2 above produces this output:

$ python my_app.py
server:
  site:
    fb:
      domain: facebook.com
    google:
      domain: google.com
  host: localhost
  port: 443

The @package directive here could be any compound key that you want. For example, using the following defaults list:

# Option 1: With a non-overridable defaults list
defaults:
  - site/fb@foo
  - site/google@bar
# Option 2: With an overridable defaults list
defaults:
  - site@foo: fb
  - site@bar: google

We get this result:

$ python my_app.py
server:
  foo:
    domain: facebook.com
  bar:
    domain: google.com
  host: localhost
  port: 443

Using option 2 (an overridable defaults list) means you can override the given default option using the CLI:

$ python my_app.py server/site@server.foo=amazon
server:
  foo:
    domain: amazon.com
  bar:
    domain: google.com
  host: localhost
  port: 443