Configuration

Basics: Overriding Configurations

You can override configurations directly from the command line by appending key=value pairs to your command. For example, if you want to change the learning rate, which is defined as follows in YAML:

# configurations/experiment/xxx.yaml
experiment:
  training:
    lr: 1e-3

you can override it with the following command:

python -m main ... experiment.training.lr=5e-3

Dataset-Dependent Configurations

In many cases, algorithm and experiment configurations are dependent on the dataset. For instance, in the video_generation experiment using the dfot_video algorithm, configurations such as batch size, learning rate, and model size vary based on the dataset. To manage this variability, we utilize Hydra's Specializing Configuration feature.

This is implemented in the configurations/dataset_experiment folder, enabling dataset-specific customization of experiment and algorithm settings. YAML files follow the naming convention {dataset_name}_{experiment_name}.yaml. If a corresponding file exists, it automatically overrides the default configurations for the dataset, algorithm, and experiment.

Shortcuts

Shortcuts simplify configuration overrides by providing a convenient alternative to lengthy command-line arguments. For instance, instead of manually specifying the size parameters (depth, num_heads, width) of a DiT model, you can use @DiT/size (e.g., @DiT/XL).

The unwrap_shortcuts function in utils/hydra_utils.py automatically expands shortcuts into command-line overrides. The placement of @shortcut determines the order of composition:

1. Command-line arguments before @shortcut
2. Shortcut overrides
3. Command-line arguments after @shortcut

Each shortcut @{name} maps to a YAML file in configurations/shortcut/{name}.yaml. For example, @DiT/XL corresponds to configurations/shortcut/DiT/XL.yaml.