I am trying to use the multirun feature of Hydra. I have the following example layout:
├── conf
│ ├── config.yaml
│ └── test
│ ├── imagenet.yaml
│ └── mnist.yaml
└── main.py
I have a hyperparameter called lambda that is tuned for each test and I want to test multiple values of this parameter. For example, I want to run the imagenet test with lambda=0 and lambda=100 but I want to run the MNIST test with lambda=0 and lambda=1.
My conf/config.yaml:
defaults:
- test: ???
- _self_
hydra:
mode: MULTIRUN
My conf/test/imagenet.yaml:
hydra:
sweeper:
params:
lambda: 0,100
foo: bar
beep: boop
My conf/test/mnist.yaml:
hydra:
sweeper:
params:
lambda: 0,1
baz: baq
meep: moop
I would like to then run python main.py test=glob(*), and for Hydra to run 4 tests: imagenet with lambda equal to 0 and 100, and MNIST with lambda equal to 0 and 1. However, Hydra does not appear to recognize parameter sweeps if they are not specified in the global config file.
How can I accomplish parameter sweeps unique to group config files?
I am trying to use the multirun feature of Hydra. I have the following example layout:
├── conf
│ ├── config.yaml
│ └── test
│ ├── imagenet.yaml
│ └── mnist.yaml
└── main.py
I have a hyperparameter called lambda that is tuned for each test and I want to test multiple values of this parameter. For example, I want to run the imagenet test with lambda=0 and lambda=100 but I want to run the MNIST test with lambda=0 and lambda=1.
My conf/config.yaml:
defaults:
- test: ???
- _self_
hydra:
mode: MULTIRUN
My conf/test/imagenet.yaml:
hydra:
sweeper:
params:
lambda: 0,100
foo: bar
beep: boop
My conf/test/mnist.yaml:
hydra:
sweeper:
params:
lambda: 0,1
baz: baq
meep: moop
I would like to then run python main.py test=glob(*), and for Hydra to run 4 tests: imagenet with lambda equal to 0 and 100, and MNIST with lambda equal to 0 and 1. However, Hydra does not appear to recognize parameter sweeps if they are not specified in the global config file.
How can I accomplish parameter sweeps unique to group config files?
Share Improve this question edited Mar 31 at 19:10 void_panda asked Mar 31 at 17:35 void_pandavoid_panda 631 silver badge5 bronze badges1 Answer
Reset to default 0For starters, sweeping config is not adding anything not supported by the command line. It's just another representation of the same capabilities.
This kind of sweep you want is not supported directly.
The basic sweeper is simply doing the outer product of all combinations.
You can just run your app twice from a script or create experiment specific files for each of your variants and sweep over those.
├── conf
│ ├── config.yaml
│ └── test
│ ├── imagenet_1.yaml
│ ├── imagenet_100.yaml
│ ├── mnist_1.yaml
│ └── mnist_100.yaml
└── main.py
Specifically, the experiment config files, you don't need to repeat the whole thing, just use the experiment config pattern.