I just started evaluating Kedro for use, and I began with a small project where I read data from a MS-SQL Server. The pipeline will run with a few months in between, and with new date ranges every time. To get this parameter (date) into the pipeline I looked into using globals.yml
. The kedro run
command works, I have created just one node that loads the data.
I load some parameters from globals.yml
to use in catalog.yml
.
If I use the standard lines in settings.py
from kedro.config import TemplatedConfigLoader
CONFIG_LOADER_CLASS = TemplatedConfigLoader
CONFIG_LOADER_ARGS = {
"globals_pattern": "*globals.yml",
}
When doing this, it does run perfectly, BUT I get a warning message, specifically this
FutureWarning: TemplatedConfigLoader
will be deprecated in Kedro 0.19. Please use the
OmegaConfigLoader instead. To consult the
documentation for OmegaConfigLoader, see here:
https://docs.kedro.org/en/stable/configuration/advanc
ed_configuration.html#omegaconfigloader
Perfectionist as I am, I want to get rid of this warning and be prepared for Kedro 0.19. Looking at the link in the warning, I replaced the above code in settings.py
to
from kedro.config import OmegaConfigLoader # new import
CONFIG_LOADER_CLASS = OmegaConfigLoader
CONFIG_LOADER_ARGS = {
"globals_pattern": "*globals.yml",
}
Then it complains about __init__() got an unexpected keyword argument 'globals_pattern'
.
OK so I try to remove the CONFIG_LOADER_ARGS
thinking it is already there by default. Does not work either.
How do I get globals.yml to work properly with OmegaConf in Kedro?
This is now supported:
If you are on 0.18.14, it's also supported in the 0.19.x versions: https://docs.kedro.org/en/0.18.14/configuration/advanced_configuration.html#how-to-use-global-variables-with-the-omegaconfigloader