OmegaConf allows you to register a custom resolver. Here is an example of resolving a tuple.
def resolve_tuple(*args):
return tuple(args)
OmegaConf.register_new_resolver("tuple", resolve_tuple)
This can be used to resolve a value in a config file with a structure like ${tuple:1,2} to a tuple (1, 2). Along with hydra.utils.instantiate this can be used to create objects that contain or utilize tuples. For example:
config.yaml
obj:
tuple: ${tuple:1,2}
test.py
import hydra
import hydra.utils as hu
from omegaconf import OmegaConf
def resolve_tuple(*args):
return tuple(args)
OmegaConf.register_new_resolver('tuple', resolve_tuple)
@hydra.main(config_path='conf', config_name='config_test')
def main(cfg):
obj = hu.instantiate(cfg.obj, _convert_='partial')
print(obj)
if __name__ == '__main__':
main()
Running this example returns:
$ python test.py
{'tuple': (1, 2)}
However, imagine you had a much more complex config structure. You may want to use interpolation to bring in configs from other files like so.
tuple/base.yaml
tuple: ${tuple:1,2}
config.yaml
defaults:
- tuple: base
- _self_
obj:
tuple: ${tuple}
Running this example you get an error:
$ python test.py
Error executing job with overrides: []
Traceback (most recent call last):
File "test.py", line 16, in main
obj = hu.instantiate(cfg.obj, _convert_='partial')
File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 175, in instantiate
OmegaConf.resolve(config)
omegaconf.errors.UnsupportedValueType: Value 'tuple' is not a supported primitive type
Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
The full traceback from hydra is:
Error executing job with overrides: []
Traceback (most recent call last):
File "test.py", line 21, in <module>
main()
File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/hydra/main.py", line 52, in decorated_main
config_name=config_name,
File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/hydra/_internal/utils.py", line 378, in _run_hydra
lambda: hydra.run(
File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/hydra/_internal/utils.py", line 214, in run_and_report
raise ex
File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/hydra/_internal/utils.py", line 211, in run_and_report
return func()
File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/hydra/_internal/utils.py", line 381, in <lambda>
overrides=args.overrides,
File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/hydra/_internal/hydra.py", line 111, in run
_ = ret.return_value
File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/hydra/core/utils.py", line 233, in return_value
raise self._return_value
File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/hydra/core/utils.py", line 160, in run_job
ret.return_value = task_function(task_cfg)
File "test.py", line 17, in main
model = hu.instantiate(cfg.obj, _convert_='partial')
File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 175, in instantiate
OmegaConf.resolve(config)
File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/omegaconf/omegaconf.py", line 792, in resolve
omegaconf._impl._resolve(cfg)
File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/omegaconf/_impl.py", line 40, in _resolve
_resolve_container_value(cfg, k)
File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/omegaconf/_impl.py", line 19, in _resolve_container_value
_resolve(resolved)
File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/omegaconf/_impl.py", line 40, in _resolve
_resolve_container_value(cfg, k)
File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/omegaconf/_impl.py", line 23, in _resolve_container_value
node._set_value(resolved._value())
File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/omegaconf/nodes.py", line 44, in _set_value
self._val = self.validate_and_convert(value)
File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/omegaconf/nodes.py", line 57, in validate_and_convert
return self._validate_and_convert_impl(value)
File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/omegaconf/nodes.py", line 134, in _validate_and_convert_impl
f"Value '{t.__name__}' is not a supported primitive type"
omegaconf.errors.UnsupportedValueType: Value 'tuple' is not a supported primitive type
If you really dig around in the omegaconf code in the trace you will find that there is a flag for the config object allow_objects that is True in the example that passes and None in the example that does not. What is interesting is that in the _instantaite2.py file just before calling Omegaconf.resolve(config) several flags are set, one being allow_objects as True.
Is the intended behavior for these interpolated/resolved values populated from separate files to override this flag? If so, is there some way to ensure that the allow_objects flag is (or remains) true for all resolved and interpolated values?
I think there is some confusion because you are using the word tuple for multiple different purposes :)
Here is an example that works for me:
# my_app.py
import hydra
import hydra.utils as hu
from omegaconf import OmegaConf
def resolve_tuple(*args):
return tuple(args)
OmegaConf.register_new_resolver('as_tuple', resolve_tuple)
@hydra.main(config_path='conf', config_name='config')
def main(cfg):
obj = hu.instantiate(cfg.obj, _convert_='partial')
print(obj)
if __name__ == '__main__':
main()
# conf/config.yaml
defaults:
- subdir: base
- _self_
obj:
a_tuple: ${subdir.my_tuple}
# conf/subdir/base.yaml
my_tuple: ${as_tuple:1,2}
$ python my_app.py # at the command line:
{'a_tuple': (1, 2)}
The main difference here is that we've got a_tuple: ${subdir.my_tuple} instead of a_tuple: ${my_tuple}.
Notes:
allow_objects flag that you mentioned is undocumented and it's behavior is subject to change.