How to gather config files in a list with Hydra-fb?

Question

Say I have an abstract class db in my code and classes db1, db1, ... db1 that inherit from db. My project uses hydra and has this structure:

├── my_app.py
├── conf.yaml
└── db
    ├── db1.yaml
    ├── db2.yaml
    └── db3.yaml

I need a list of db so I would like to get a final configuration file like this :

db:
  -
    param1_of_db1: key_1_1
    param2_of_db1: key_1_2
  -
    param1_of_db2: key_2_1
    param2_of_db2: key_2_2
  -
    param1_of_db3: key_3_1
    param2_of_db3: key_3_2

so that db is a list of params of db1, db2, db3. In conf.yaml file, I imagine something like:

defaults:
  - db: [db1, db2, db3]

Is there a way to do something like that ?

Answer 1

What you are asking for is not supported by Hydra.

1. List composition is not supported, lists are all or nothing during composition.
2. Config groups are mutually exclusive, there is a feature request to relax that.

You can get close to it though (without being able to override the structure from the command line is something like: config.yaml:

defaults:
  - db/db1
  - db/db2
  - db/db3

This syntax is documented here.

In each db config file you can do something like:

db/db1.yaml:

# @pacakge _group_._name_
host: localhost
port: 3306

Package overrides are documented here.

The resulting config would looke like:

db:       # from the config group of the corresponding config (path)
  db1:    # from the name of the corresponding config
    host: localhost
    port: 3306
  db2:
    ...