Allow multiple static inputs #822
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This property was unused and had a type inconsistency (returning torch.tensor([]) instead of list[torch.Tensor] for the empty case). Co-authored-by: Cursor <cursoragent@cursor.com>
Add a `static_inputs: dict[str, str] | None` field to TrainerConfig that maps variable names to file paths. The build method now constructs StaticInputs from this config rather than inferring topography from the fine training data path. Co-authored-by: Cursor <cursoragent@cursor.com>
…nputs - Rename `Topography` class to `StaticInput` and export it publicly - Update `GriddedData` and `PairedGriddedData` to store `static_inputs: StaticInputs` instead of `topography: Topography` - Update `_get_input_from_coarse` in DiffusionModel to accept StaticInputs and loop over all fields rather than a single topography - Rename `build_topography` to `build_static_inputs` in DataLoaderConfig - Update `PairedDataLoaderConfig.build` to pass StaticInputs - Remove unused `get_topography` from CheckpointModelConfig Co-authored-by: Cursor <cursoragent@cursor.com>
Rename topography -> static_inputs throughout the codebase: - train.py: training and validation loop variables - predict.py: downscaler and event downscaler - evaluator.py: evaluator and event evaluator - inference/: downscaler, output config, work items - _deterministic_models.py: model methods - predictors/cascade.py: cascade predictor and config - predictors/composite.py: patch predictor Co-authored-by: Cursor <cursoragent@cursor.com>
Update all test files to use StaticInput/StaticInputs instead of Topography: - test_topography.py: rename Topography -> StaticInput in test cases - test_config.py: pass StaticInputs to DataLoaderConfig.build - test_patching.py: use StaticInputs wrapper for patching tests - test_models.py: update model train/generate tests - test_predict.py: update predictor integration tests - test_train.py: add static_inputs config to test setup - test_train_config.yaml: add static_inputs field - test_inference.py: update inference/downscaler tests - test_output.py: pass StaticInputs to output config build - test_cascade.py: update cascade predictor tests - test_composite.py: update composite predictor tests Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This refactors the downscaling code to use the full
StaticInputsclass ingeneratecalls.DiffusionModel._get_input_from_coarseconcats allStaticInputs.fields.dataas additional input channelsTrainConfighas a new fieldstatic_inputswhich should be a mapping of{var_name: path_to_dataset}. This should be used to provide the static inputs fields- they will no longer be loaded automatically from the fine dataset. In the future,use_fine_topographyin the config should be deprecated since thestatic_inputsfield is the source of this information.GriddedData, PairedGriddedDataare updated to have aStaticInputsattribute instead of the singleTopographytopographyare updated to the more generalstatic_inputtrain, predict, inference, evaluatorentrypoint code and their tests are updated to pass theStaticInputsto generate calls(Exact diff of v1 branch but used cursor to organize the commits to be more reviewable)