Python hydra sweeper parameter treated as string

Question

I want to do a hydra sweep, where I sweep over possible Python functions to instantiate with hydra.utils.call (or hydra.utils.instantiate). When I do the sweep however, hydra thinks the parameters are a string, instead of an object to be instantiated within the Python main script.

I have a directory project_root with 6 files:

├── config
│   ├── main.yaml
│   └── optimization
│       ├── main.yaml
│       └── utility_function
│           ├── eig.yaml
│           └── ucb.yaml
├── main.py
└── utils.py

The main config file project_root/config/main.yaml has:

defaults:
  - _self_
  - optimization: main

hydra:
  sweeper:
    params:
      optimization.utility_function: eig, ucb

The optimization default comes from project_root/optimization/main.yaml:

defaults:
  - utility_function: eig

In the sweeper, eig and ucb point to Python functions, for example project_root/config/optimization/utility_function/eig.yaml is:

_target_: utils.eig
_partial_: true

And similarly, project_root/config/optimization/utility_function/ucb.yaml is:

_target_: utils.ucb
_partial_: true

The _partial_ key is set based on what I saw in the documentation, since I want to pass values to the instantiated Python functions in the real project I'm working on.

Both of the above refer to Python functions called eig and ucb in project_root/utils.py:

def eig(x):
    return "This is the EIG function"


def ucb(x):
    return "This is the UCB function"

All of this is brought together in project_root/main.py:

import hydra
from hydra.utils import call


def train_debug(cfg):
    print(f"cfg.optimization.utility={cfg.optimization.utility_function}.")
    print(f"type(cfg.optimization.utility)={type(cfg.optimization.utility_function)}.")
    print(call(cfg.optimization.utility_function)("dummyarg"))


@hydra.main(config_path="config", config_name="main", version_base="1.3")
def run(cfg):
    train_debug(cfg)


if __name__ == "__main__":
    run()

When I want to do the sweep with:

python main.py -m

I get the following output:

[2023-06-21 14:58:04,932][HYDRA] Launching 2 jobs locally
[2023-06-21 14:58:04,932][HYDRA]    #0 : optimization.utility_function=eig
cfg.optimization.utility=eig.
type(cfg.optimization.utility)=<class 'str'>.
[2023-06-21 14:58:05,000][HYDRA]    #1 : optimization.utility_function=ucb
cfg.optimization.utility=ucb.
type(cfg.optimization.utility)=<class 'str'>.
Error executing job with overrides: ['optimization.utility_function=eig']
Cannot instantiate config of type str.
Top level config must be an OmegaConf DictConfig/ListConfig object,
a plain dict/list, or a Structured Config class or instance.

Things do work when I just run:

python main.py

And I get the output:

cfg.optimization.utility={'_target_': 'utils.eig', '_partial_': True}.
type(cfg.optimization.utility)=<class 'omegaconf.dictconfig.DictConfig'>.
This is the EIG function

Any ideas on what I should do to be able to do a sweep over the functions?

Answer 1

This means that the key optimization.utility_function is getting the string values eig and ucb in each of the runs.

hydra:
  sweeper:
    params:
      optimization.utility_function: eig, ucb

The proper way to override nested config groups is with a slash as separator. Also, before you use the sweeper config, I suggest you try in the command line. Something like:

$ python main.py -m optimization/utility_function=eig,ucb