Search code examples
fb-hydra

Hydra omegaconf resolvers on dictionaries and lists

I'm trying to use omegaconf resolvers on dictionaries and lists but I keep getting an error. for example the following code gives a hydra.errors.OverrideParseException

(taken from here Hydra combine lists in config)

(I'm running on Hydra 1.1.2)

app.py:

import hydra
from omegaconf import OmegaConf
# custom list merge resolver
OmegaConf.register_new_resolver("merge", lambda x, y : x + y)

@hydra.main(config_name="test.yaml", config_path="./")
def main(cfg):
    OmegaConf.resolve(cfg)
    print(cfg)

if __name__ == "__main__":
    main()

Config file test.yaml:

list1 : [0]
list2 : [1,2,3]

list3 : ${merge:${list1},${list2}}

Traceback (most recent call last):
File "C:\venvs\color-correction\lib\site-packages\hydra\_internal\utils.py", line 377, in _run_hydra
    run_and_report(
  File "C:\venvs\color-correction\lib\site-packages\hydra\_internal\utils.py", line 214, in run_and_report
    raise ex
  File "C:\venvs\color-correction\lib\site-packages\hydra\_internal\utils.py", line 211, in run_and_report
    return func()
  File "C:\venvs\color-correction\lib\site-packages\hydra\_internal\utils.py", line 378, in <lambda>
    lambda: hydra.run(
  File "C:\venvs\color-correction\lib\site-packages\hydra\_internal\hydra.py", line 88, in run
    cfg = self.compose_config(
  File "C:\venvs\color-correction\lib\site-packages\hydra\_internal\hydra.py", line 559, in compose_config
    cfg = self.config_loader.load_configuration(
  File "C:\venvs\color-correction\lib\site-packages\hydra\_internal\config_loader_impl.py", line 141, in load_configuration
    return self._load_configuration_impl(
  File "C:\venvs\color-correction\lib\site-packages\hydra\_internal\config_loader_impl.py", line 231, in _load_configuration_impl
    parsed_overrides = parser.parse_overrides(overrides=overrides)
  File "C:\venvs\color-correction\lib\site-packages\hydra\core\override_parser\overrides_parser.py", line 96, in parse_overrides
    raise OverrideParseException(
hydra.errors.OverrideParseException: Error parsing override 'list3=${merge:${list1},${list2}}'
extraneous input '}' expecting <EOF>
See https://hydra.cc/docs/next/advanced/override_grammar/basic for details
python-BaseException

I'm trying to use omegaconf resolvers on dictionaries and lists.

Solution

  • When passing values such as interpolations via the CLI, it is sometimes necessary to use an extra set of quotes.

    Instead of running python app.py 'list3=${merge:${list1},${list2}}', run python app.py 'list3="${merge:${list1},${list2}}"'.

    See the hydra docs on quoting at the CLI for more information.