Search code examples
pythontensorflowkeraswandb

Wandb ignores --configs flag when running training script

Running a script I want to specify what config YAML to use. The documentation states you can specify what configuration file to load using the --configs flag, however running

>>> python myscript.py --configs new-configs.yaml

on the following script

import wandb
wandb.init()
print(wandb.config)

I can see that the configuration contained in config-defaults.yaml is being loaded instead. Why is this happening?

Specs

MacOS 13.4.1 (Apple M1 chip)
python                        3.10.12
tensorflow-macos              2.12.0
tensorflow-metal              0.8.0
wandb                         0.15.5
Solution

  • The --configs flag is used to specify a different configuration file other than the default config-defaults.yaml. However, it's important to note that the --configs flag should be used when running the wandb command, not when running your python script. If you want to use a different configuration file when running your python script, you should load the configuration from the file manually in your script. Here is an example of how you can do this:

    python
import wandb
import yaml

# Load config from file
with open('new-configs.yaml') as file:
    config = yaml.safe_load(file)

# Initialize wandb with the loaded config
wandb.init(config=config)

print(wandb.config)

    In this example, the configuration is loaded from new-configs.yaml using the yaml library, and then passed to wandb.init().

    If you still want to use the --configs flag, you should use it with the wandb command, like this:

    bash
wandb --configs=new-configs.yaml

    This will start a new run with the configuration from new-configs.yaml. However, this will not run your python script. If you want to run your script with this configuration, you should load the configuration from wandb.config in your script, like this:

    python
import wandb

# Initialize wandb
wandb.init()

# Get the configuration
config = wandb.config

print(config)

    In this example, the configuration is loaded from wandb.config, which is set by the wandb command with the --configs flag.

    I hope this helps!