Skip to content

Conversation

@jonasbhend
Copy link
Contributor

@jonasbhend jonasbhend commented Jan 15, 2026

This PR supports generation of ICON-based baseline datasets for evaluation with evalml.

Due to snakemake default behaviour of deleting partial (unfinished) results, extracting large zarr datasets via snakemake is not the ideal fit. Therefore, this PR also removes the corresponding rules hinting at using snakemake for dataset extraction. Instead, datasets are extracted using the python script directly.

Changes

  • Add support for extraction of baseline data from 'live' archives (i.e. not tared and with different file structure compared to archived COSMO NWP data).
  • remove extract rules (data.smk)

@jonasbhend jonasbhend marked this pull request as ready for review January 15, 2026 14:25
@jonasbhend jonasbhend requested a review from frazane January 15, 2026 14:31
jonasbhend and others added 2 commits January 16, 2026 10:05
Co-authored-by: Francesco Zanetta <62377868+frazane@users.noreply.github.com>
@jonasbhend jonasbhend requested a review from frazane January 16, 2026 09:08
Copy link
Member

@dnerini dnerini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking good thanks!

@jonasbhend jonasbhend force-pushed the MRB-499-Extract-baseline-for-evaluation-ICON-CH1-EPS branch from b91abd4 to a2ff3fe Compare January 29, 2026 12:48
@jonasbhend jonasbhend merged commit c3834b2 into main Jan 29, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants