Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
324 commits
Select commit Hold shift + click to select a range
e70bfa3
fix: enable thread-safety in h5py build for Docker test environment
pgierz Nov 12, 2025
b12c852
test: add comprehensive h5py thread-safety verification
pgierz Nov 12, 2025
44aaaf5
test: expand h5py thread-safety tests to cover all backends and scena…
pgierz Nov 12, 2025
1974880
fix: correct h5py thread-safety verification method
pgierz Nov 12, 2025
2358d8c
fix: switch to netcdf4 engine with parallel processing enabled
pgierz Nov 12, 2025
a9de50f
test: skip h5netcdf tests with parallel=True to prevent segfaults
pgierz Nov 12, 2025
d7ddf58
fix: disable parallel file opening due to thread-safety requirements
pgierz Nov 12, 2025
aed131c
fix: switch to micromamba for compatible HDF5/NetCDF stack
pgierz Nov 12, 2025
135ab42
fix: resolve license deprecation warnings and Docker permissions
pgierz Nov 12, 2025
c824d1a
fix: ensure workspace has proper permissions for coverage files
pgierz Nov 12, 2025
8d58abe
fix: run Docker containers as root to fix coverage permissions
pgierz Nov 12, 2025
c4adbb1
feat: restructure xarray config with nested YAML support and auto-inj…
pgierz Nov 12, 2025
cedc6bd
style: apply black formatting to config.py
pgierz Nov 12, 2025
a1155a2
fix: correct doctest example in config_injector
pgierz Nov 12, 2025
c4de102
Merge PR #230: Add CMIP7 Support - Global Attributes
pgierz Nov 12, 2025
942422b
test: remove xfail markers from CMIP7 integration tests
pgierz Nov 12, 2025
d674b2b
fix: resolve merge conflict in CI workflow file
pgierz Nov 12, 2025
6051349
style: apply black and isort formatting to CMIP7 files
pgierz Nov 12, 2025
0e160d8
feat: add cmip7 optional dependency and update Docker
pgierz Nov 12, 2025
dd83d1d
feat: implement automatic metadata generation in CMIP7Interface
pgierz Nov 13, 2025
a886af3
fix: correct export_dreq_lists_json CLI args and update doctest
pgierz Nov 13, 2025
a8b202b
docs: add working examples to CMIP7Interface module docstring
pgierz Nov 13, 2025
3628932
fix: return integer types for variant label index attributes
pgierz Nov 13, 2025
52e3db1
fix: use packaged data for CMIP7 table_dict_from_directory
pgierz Nov 13, 2025
f066a78
style: remove unused logger import from table.py
pgierz Nov 13, 2025
15a0011
fix: standardize variant index attributes to strings for CMIP6 and CMIP7
pgierz Nov 13, 2025
3eb9e2c
test: suppress noisy distributed/dask/prefect logs in pytest
pgierz Nov 13, 2025
4c81801
fix: properly suppress distributed/dask/prefect logs in pytest
pgierz Nov 13, 2025
c304a2f
fix: use autouse fixture for log suppression instead of pytest_configure
pgierz Nov 13, 2025
831ee92
Apply suggestions from code review
siligam Nov 13, 2025
0b1e542
refactored chunk functions
siligam Nov 13, 2025
eed4d3a
fix: use LocalCluster silence_logs parameter to suppress worker logs
pgierz Nov 13, 2025
aec3349
Merge branch 'prep-release' into feat/chunks
siligam Nov 13, 2025
f04c881
fix: add cmip7_data_request_dir fixture to setup CMIP7 test data
pgierz Nov 14, 2025
72391e3
fix: add -a flag to export_dreq_lists_json command
pgierz Nov 14, 2025
288c0c7
fix: use get_variables_metadata to generate CMIP7 variable metadata
pgierz Nov 14, 2025
b86a0d6
ci: add persistent GitHub Actions cache for test data
pgierz Nov 14, 2025
8b04b9a
fix: improve error reporting in test data download fixtures
pgierz Nov 14, 2025
8334ec2
feat: implement stub data system for lightweight CI testing
pgierz Nov 14, 2025
de3455a
fix: exclude auto-generated stub manifests from yamllint
pgierz Nov 14, 2025
275d470
fix: skip meta tests when using stub data
pgierz Nov 14, 2025
6f09599
fix: use lazy fixture loading to prevent unwanted downloads
pgierz Nov 14, 2025
3b22579
fix: generate unique time coordinates for stub NetCDF files
pgierz Nov 14, 2025
f85239c
fix: make CMIP_Tables_Dir optional for CMIP7 configurations
pgierz Nov 14, 2025
f148146
fix: use safe dictionary access for cmip6_cmor_table key
pgierz Nov 14, 2025
faef9b9
feat: generate minimal FESOM mesh files for stub testing
pgierz Nov 14, 2025
9b5fe0a
fix: create mesh files directly in stub fixtures
pgierz Nov 14, 2025
c9777dc
fix: add missing FESOM mesh files (aux3d, elem3d, depth)
pgierz Nov 14, 2025
bb035af
test: mark FESOM regridding tests to require real data
pgierz Nov 14, 2025
ba3aff0
test: skip FESOM tests when using stub data
pgierz Nov 14, 2025
825861b
feat: use git-lfs to clone FESOM PI mesh from GitLab
pgierz Nov 14, 2025
8875306
feat: enable real test data in CI and add mesh test skipping
pgierz Nov 14, 2025
4f4f204
fix: add git-lfs to Docker test image
pgierz Nov 18, 2025
bbb08e5
fix: pre-generate CMIP7 metadata and add cache lookup for doctests
pgierz Nov 18, 2025
bd79c92
fix: use correct export_dreq_lists_json command for CMIP7 metadata
pgierz Nov 18, 2025
b181b00
fix: check multiple cache locations for CMIP7 metadata
pgierz Nov 18, 2025
ae31f48
fix: use PYCMOR_CMIP7_METADATA_DIR env var for cache location
pgierz Nov 18, 2025
80df943
fix: use correct flags for export_dreq_lists_json fallback
pgierz Nov 18, 2025
09ea744
chore: clean up badly committed file
pgierz Nov 18, 2025
9adebba
chore: better ignore patterns
pgierz Nov 18, 2025
8da66cd
fix: break long line for flake8 compliance
pgierz Nov 18, 2025
5ab7315
fix: make load_metadata() a side-effect-only method
pgierz Nov 19, 2025
466bf8e
fix: bad strings in doctest
pgierz Nov 19, 2025
004201f
test: removes old integration test for investigating segfault, no lon…
pgierz Nov 19, 2025
b1577d8
wip [skip ci]
pgierz Nov 19, 2025
7bdd0bd
tests: allows for env variable of test data cache
pgierz Nov 19, 2025
360f666
test: skips bad pyfesom2 tests
pgierz Nov 19, 2025
7b0d580
test: add CMIP7 parameterization for integration tests
pgierz Nov 19, 2025
ac07841
chore: update black version in pre-commit config to support py312
pgierz Nov 19, 2025
c97a16d
ci: remove unnecessary package installation from lint job
pgierz Nov 19, 2025
68ea78d
fix: remove unused pytest import in test_uxarray_pi
pgierz Nov 19, 2025
32303eb
chore: end-of-file fixer from precommit
pgierz Nov 19, 2025
6531bb8
ci: sanitize branch names for Docker image tags
pgierz Nov 19, 2025
55305c9
fix: add CV_Dir workaround for CMIP7 configs
pgierz Nov 19, 2025
15fd8dc
util: adds testing script
pgierz Nov 19, 2025
732f313
Merge pull request #237 from esm-tools/test/integration/cmip7-paramet…
pgierz Nov 19, 2025
40f7671
wip
pgierz Nov 19, 2025
c0fe8d1
examples: fix CMIP7 conversion for example 01 (unit conversion)
pgierz Nov 19, 2025
5369e7f
refactor: implement 5-level priority-based resource loader system
pgierz Nov 19, 2025
1ac8630
test: add comprehensive unit tests for ResourceLoader system
pgierz Nov 19, 2025
4aa96a5
docs: update CMIP7 configuration docs for ResourceLoader system
pgierz Nov 19, 2025
9a6bc1a
fix: update CMIP7 validation tests for optional CV_Dir
pgierz Nov 19, 2025
c3cd284
Merge remote-tracking branch 'origin/refactor/resource-loader' into e…
pgierz Nov 20, 2025
5eeff64
wip
pgierz Nov 20, 2025
c91172e
refactor: use factory pattern for CVLoader with class-level config
pgierz Nov 20, 2025
bc3ba40
feat: Add CF-compliant coordinate metadata with YAML externalization
siligam Nov 20, 2025
cddfb3c
feat: Add validation for existing coordinate metadata
siligam Nov 20, 2025
489d48d
docs: Add comprehensive RST documentation for coordinate attributes
siligam Nov 20, 2025
4daf647
style: Fix isort and black formatting for CI
siligam Nov 21, 2025
3476503
feat: Add dimension mapping from source data to CMIP standards
siligam Nov 21, 2025
f1c96e9
docs: Add comprehensive RST documentation for dimension mapping
siligam Nov 21, 2025
7edb261
feat: Add per-rule dimension override capability
siligam Nov 21, 2025
8731ddd
feat: Add tests and documentation for dimension override feature
siligam Nov 21, 2025
355a3ba
style: Apply black formatting to test_dimension_mapping.py
siligam Nov 21, 2025
8d5028f
Refactor chunking config to use inherit block
siligam Nov 21, 2025
7066a37
docs: Update netcdf_chunking.rst title per review
siligam Nov 21, 2025
9da325b
wip
pgierz Nov 21, 2025
741af87
refactor: rename ResourceLoader to ResourceLocator for semantic clarity
pgierz Nov 22, 2025
fb9cc9e
fix: add MetaFactory metaclass to locator base classes
pgierz Nov 22, 2025
64a0c74
wip
pgierz Nov 22, 2025
2106823
fix: add logging options to process command
pgierz Nov 22, 2025
6f4d449
fix: replace deprecated pkg_resources with importlib.metadata
pgierz Nov 22, 2025
76a4602
fix: suppress pint unit redefinition warnings
pgierz Nov 22, 2025
1e284c3
wip...
pgierz Nov 22, 2025
8dfec19
fix: properly suppress pint warnings after cf_xarray import
pgierz Nov 22, 2025
9cb59a4
fix: suppress pint warnings by temporarily raising log level
pgierz Nov 22, 2025
c71f926
fix: replace pkg_resources with importlib.metadata in utils.py
pgierz Nov 22, 2025
4abe088
fix: show full warning context in logging output
pgierz Nov 22, 2025
7488a9d
wip
pgierz Nov 22, 2025
55bd510
wip
pgierz Nov 22, 2025
8a0d6e9
feat: add ASCII banner and improve logging throughout application
pgierz Nov 22, 2025
52bc52c
wip
pgierz Nov 22, 2025
4fd693d
fix: add --recurse-submodules to git clone and improve debug logging
pgierz Nov 22, 2025
7dce3dd
fix: add REPO_SUBDIR for repositories with subdirectories
pgierz Nov 22, 2025
7fd65d2
fix: merge inherit values into rules before validation
pgierz Nov 23, 2025
6ec19bb
feat: derive table_id from compound_name when not explicitly provided
pgierz Nov 23, 2025
b8b5641
better cleanup
pgierz Nov 23, 2025
d14183c
debug: add logging to table_id derivation logic
pgierz Nov 23, 2025
cda704b
Add compound_name to global_attributes_set_on_rule attrs tuple
pgierz Nov 23, 2025
35df7b9
wip: wtf? removes old pymor names from tests
pgierz Nov 23, 2025
713dde7
wip: fix bad pipeline/get
pgierz Nov 23, 2025
f442493
wip
pgierz Nov 23, 2025
5173469
wip
pgierz Nov 23, 2025
f2a31b6
Merge pull request #240 from esm-tools/examples/cmip7-01
pgierz Nov 23, 2025
2469fe9
wip for example 2
pgierz Nov 23, 2025
3dbd6b3
wip
pgierz Nov 23, 2025
7eb7b7a
style: messed up yaml trailing space, precommit not on levante
pgierz Nov 23, 2025
3c16d37
wip for ex3
pgierz Nov 23, 2025
c371394
Merge pull request #241 from esm-tools/examples/cmip7-02
pgierz Nov 23, 2025
7e2c1e6
wip
pgierz Nov 23, 2025
8d7038c
wip: skip dimensionless unit check if model unit is specifically give…
pgierz Nov 23, 2025
f3d2711
wip
pgierz Nov 23, 2025
ce9f603
wip for ex 4
pgierz Nov 23, 2025
a963ba1
cmip7 seems to work too
pgierz Nov 23, 2025
518a4c7
Merge pull request #242 from esm-tools/examples/cmip7-03
pgierz Nov 23, 2025
8551d46
Merge branch 'prep-release' into feat/chunks
pgierz Nov 23, 2025
5578cb1
removes badly commited output file (?)
pgierz Nov 23, 2025
52a9ba6
Merge pull request #243 from esm-tools/examples/cmip7-04
pgierz Nov 23, 2025
03683a6
back-merge prep-release
pgierz Nov 23, 2025
ec791d0
wip
pgierz Nov 23, 2025
84c5a86
chore: clean out old file
pgierz Nov 23, 2025
33f16ed
chore: clean out old file
pgierz Nov 23, 2025
3810851
chore: clean out old file
pgierz Nov 23, 2025
0e98d4c
Merge branch 'prep-release' into feat/chunks
pgierz Nov 23, 2025
4c92626
wip: better converter for potential missing name attr for dataarray -…
pgierz Nov 23, 2025
a1bdb72
Merge pull request #233 from esm-tools/feat/chunks
pgierz Nov 23, 2025
f1f7de1
Apply suggestions from code review
pgierz Nov 23, 2025
0911856
Merge branch 'prep-release' into main
pgierz Nov 23, 2025
f740066
style: black
pgierz Nov 23, 2025
269140b
feat: create dedicated accessors module for xarray accessor registration
siligam Aug 2, 2025
b47d055
Add unified pymor accessor and comprehensive tests
siligam Aug 2, 2025
e01e27d
added documentation
siligam Aug 2, 2025
41a2f76
isort
siligam Aug 2, 2025
f9fbb90
updated Accessor methods section documentation in infer_freq.rst
siligam Aug 23, 2025
da12e84
fixed tests to update the FrequencyResult as return object rather tha…
siligam Aug 23, 2025
aab1300
fix typo
siligam Aug 23, 2025
428ce2d
fix: update test_accessors.py to use pycmor instead of pymor
pgierz Nov 24, 2025
8fc0df7
wip: some old names
pgierz Nov 24, 2025
7987207
enables ci for prototype branches
pgierz Nov 24, 2025
1fcc17e
Merge pull request #246 from esm-tools/feat/accessors-rebased
siligam Nov 26, 2025
420235b
Merge remote-tracking branch 'mzapponi/main' into mzapponi-merge
pgierz Nov 27, 2025
887ce91
chore(rule.py): minor cleanup in global_attrs_on_rule
pgierz Nov 27, 2025
cb1b8e6
Merge pull request #249 from esm-tools/mzapponi-merge
pgierz Nov 27, 2025
824b496
Fix linting and formatting issues
siligam Nov 27, 2025
b3e3532
Update src/pycmor/core/config.py
siligam Nov 30, 2025
a64df1f
Update src/pycmor/core/config.py
siligam Nov 30, 2025
3b5654a
Update src/pycmor/core/pipeline.py
siligam Nov 30, 2025
b8beffc
Update src/pycmor/std_lib/coordinate_attributes.py
siligam Nov 30, 2025
83832a3
Update src/pycmor/core/pipeline.py
siligam Nov 30, 2025
3093407
Update src/pycmor/std_lib/__init__.py
siligam Nov 30, 2025
eb19fe6
comments on coordinates attributes are addressed.
siligam Dec 1, 2025
fa3db4b
Fixed lint and format issues
siligam Dec 1, 2025
4d54493
wip.. fixing doc tests
siligam Dec 1, 2025
a37c389
chore: style
pgierz Dec 2, 2025
e52b110
wip, most of the accessor
pgierz Dec 2, 2025
ce98d0a
wip: doctests should pass now, turns off logger in some places for do…
pgierz Dec 3, 2025
f584916
chore: style on a missing file
pgierz Dec 3, 2025
319a6be
wip: dummy change A
pgierz Dec 3, 2025
fd46f7f
Merge remote-tracking branch 'origin/prep-release' into feat/coordina…
pgierz Dec 3, 2025
3d3ae10
wip: backmerge prep release and some cleaning
pgierz Dec 3, 2025
3376596
cleanup old markdown stuff
pgierz Dec 3, 2025
b6107d1
Merge pull request #239 from esm-tools/feat/coordinate-metadata
pgierz Dec 3, 2025
7521faa
wip
pgierz Dec 12, 2025
80223e2
wip for cli debugger
pgierz Dec 12, 2025
0d98352
wip
pgierz Dec 12, 2025
8ff2dee
...
pgierz Dec 12, 2025
89955b2
...
pgierz Dec 12, 2025
8628d57
better short name logic
pgierz Jan 15, 2026
9fe6a4b
isort
pgierz Jan 15, 2026
7a7d262
better validators
pgierz Jan 19, 2026
0513d32
feat: accessor API with lazy registration, StdLibAccessor, and .proce…
pgierz Mar 27, 2026
3d950a9
Merge remote-tracking branch 'origin/prep-release' into fix/gh250
pgierz Mar 27, 2026
0c78b9a
style: fix trailing whitespace and isort
pgierz Mar 27, 2026
492ca1c
fix: update pi_uxarray download URL (old Nextcloud share expired)
pgierz Mar 27, 2026
3b2af74
fix: add compound_name to CMIP7 test configs and setuptools to Docker
pgierz Mar 27, 2026
fa75da0
fix: use CMIP6-style compound names (Table.variable) in CMIP7 test co…
pgierz Mar 27, 2026
0ea98cf
fix: guard pyfesom2 imports in tests to avoid collection errors
pgierz Mar 27, 2026
66cf054
fix: guard pyfesom2 import in regridding.py for environments without …
pgierz Mar 27, 2026
11f5bbe
fix: skip pyfesom2-dependent tests when pyfesom2 is not importable
pgierz Mar 27, 2026
b405898
fix: CMIP7 compound_name matching when DRV has plain variable_id
pgierz Mar 27, 2026
ab87d7b
fix: propagate pipeline/flow errors instead of silently logging them
pgierz Mar 27, 2026
3345d7e
fix: remove raw convert() from DefaultPipeline steps
pgierz Mar 27, 2026
3c8d3f5
fix: ConfigurationError from dict default and missing activity_id
pgierz Mar 27, 2026
197802c
fix: dimension_mapping Mock iteration error and doctest in base_model…
pgierz Mar 27, 2026
ff766ee
fix: fall back to _pycmor_cfg for dimension_mapping when rule attr is…
pgierz Mar 27, 2026
51a00e1
fix: derive table_id from CMIP6-style compound names (Table.variable)
pgierz Mar 30, 2026
6ffc348
fix: filter None values from global attributes before applying to dat…
pgierz Mar 30, 2026
8f8f7a5
fix: pin sphinx<9 for sphinx_toolbox compatibility
pgierz Mar 30, 2026
130e897
feat: rename non-standard time dimension on load (OpenIFS support)
pgierz Mar 30, 2026
049a206
Merge PR #256: Accessor API, lazy registration, CMIP7 fixes
pgierz Mar 30, 2026
5115ea2
fix: CI workflow improvements and minor code fixes
pgierz Mar 27, 2026
8f3d8f4
fix: unwrap Prefect State objects to actual results in parallel proce…
pgierz Mar 30, 2026
3331d4c
fix: save_dataset returns the dataset instead of None after writing t…
pgierz Mar 30, 2026
084550c
feat: modernize test infrastructure with entry-point model discovery
pgierz Mar 27, 2026
3ee01f9
feat: add tutorial dataset system with xarray.tutorial-like API
pgierz Mar 27, 2026
0c64694
fix: handle absolute symlinks in tarballs on Python 3.12+ (tarfile fi…
pgierz Mar 30, 2026
0ab52d4
fix: convert loader doctests to code-blocks and update pi_uxarray URL…
pgierz Mar 30, 2026
ba35add
fix: use pycmor.enable_xarray_accessor() instead of pycmor.accessors …
pgierz Mar 30, 2026
969e328
fix: use valid CMIP6 source_id (AWI-ESM-1-1-LR instead of FESOM)
pgierz Mar 30, 2026
690e750
fix: unpin external test data package and use configs dict in accesso…
pgierz Mar 30, 2026
8456e44
fix: Python 3.9 entry_points() compatibility in CLI (cherry-pick from…
pgierz Mar 30, 2026
081502e
fix: extract tarballs into cache_dir directly to avoid double nesting
pgierz Mar 30, 2026
c4be8f9
fix: add PYCMOR_FORCE_REEXTRACT env var to bypass stale extraction cache
pgierz Mar 30, 2026
f4a5b15
fix: tutorial data_fetcher also had double-nesting extraction bug
pgierz Mar 30, 2026
71efbd6
debug: add logging to get_variable to trace model_variable value
pgierz Mar 30, 2026
966170e
debug: run only fesom_2p6-native-nodask-cmip6 with model_variable tra…
pgierz Mar 30, 2026
ebc26d0
ci: restore full integration tests with PYCMOR_FORCE_REEXTRACT on all…
pgierz Mar 30, 2026
9e5964a
ci: trigger rebuild to pick up fesom_dev config fixes from external p…
pgierz Mar 30, 2026
6be2697
fix: accept DataArray results in test_library_process assertions
pgierz Mar 31, 2026
e8a812c
fix: pin pycmor-test-data-fesom to v0.2.0
pgierz Mar 31, 2026
105475c
fix: set output_directory on per-rule level in test_cli_process
pgierz Mar 31, 2026
61c5b55
Merge pull request #253 from esm-tools/future/feat/test-infrastructure
pgierz Mar 31, 2026
b4b182a
Merge branch 'prep-release' into future/feat/ci-and-fixes
pgierz Apr 1, 2026
d60e521
wip: no double docker
pgierz Apr 1, 2026
bdc502f
wip: maybe fix for passing back State, not sure
pgierz Apr 1, 2026
150fbde
wip: still trying, better tests reveal small bugs
pgierz Apr 1, 2026
8fb2777
wip: better try to prefect checkpoints
pgierz Apr 1, 2026
cd2ccaa
wip: better caching for dockerfile test
pgierz Apr 1, 2026
e592022
wip: pandas change
pgierz Apr 1, 2026
4f7476e
wip: fix deprecation package not included
pgierz Apr 1, 2026
2580ff9
wip: removes insane doctest sanity check
pgierz Apr 1, 2026
91f8fa9
wip: YE vs Y
pgierz Apr 1, 2026
e232433
I never know if I should commit this one or not
pgierz Apr 1, 2026
76a0398
fix: incorrect expected doctest result
pgierz Apr 1, 2026
04d7a94
wip: doctest/freq again
pgierz Apr 1, 2026
3bf9558
Merge pull request #255 from esm-tools/future/feat/ci-and-fixes
pgierz Apr 1, 2026
02d89e6
feat: add time bounds support for CMIP-compliant datasets
pgierz Apr 1, 2026
8484933
fix: widen monthly frequency detection tolerance for time bounds
pgierz Apr 1, 2026
22daad5
Merge pull request #264 from esm-tools/feat/time-bounds
pgierz Apr 1, 2026
9647ee2
Update CI-test.yaml
pgierz Apr 3, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions .flake8
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
[flake8]
max-line-length = 120
extend-ignore = E203,W503
exclude =
.git,
__pycache__,
build,
dist,
.eggs,
*.egg-info,
_version.py,
versioneer.py,
cmip6-cmor-tables,
CMIP7_DReq_Software,
CMIP7-CVs,
src/pycmor/webapp.py
per-file-ignores =
*/_version.py:E203
2 changes: 2 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -1 +1,3 @@
src/pymor/_version.py export-subst
# SCM syntax highlighting & preventing 3-way merges
pixi.lock merge=binary linguist-language=YAML linguist-generated=true
115 changes: 115 additions & 0 deletions .github/scripts/discover_models.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
#!/usr/bin/env python
"""Discover registered model runs and orchestrator configs for GitHub Actions matrix.

This script outputs JSON for both models and orchestrator configurations that
can be used in a GitHub Actions matrix strategy.

This script only reads entry point names without loading the classes, so it
doesn't require test dependencies to be installed.
"""

import importlib.metadata
import json
import sys


def discover_model_names():
"""Discover registered model run names from entry points.

Returns just the entry point names without loading the actual classes,
so this works even without test dependencies installed.
"""
# Python 3.9 vs 3.10+ compatibility
try:
# Try Python 3.10+ API first
eps = importlib.metadata.entry_points(group="pycmor.fixtures.model_runs")
except TypeError:
# Fall back to Python 3.9 API
all_eps = importlib.metadata.entry_points()
eps = all_eps.get("pycmor.fixtures.model_runs", [])

# Just get the names, don't load the classes
model_names = [ep.name for ep in eps]

return model_names


def get_orchestrator_configs():
"""Get orchestrator configurations for the test matrix.

Returns the standard orchestrator configurations used in integration tests.
This is the single source of truth for orchestrator combinations.
"""
return [
{
"id": "prefect-dask",
"pipeline_workflow_orchestrator": "prefect",
"enable_dask": "yes",
},
{
"id": "native-dask",
"pipeline_workflow_orchestrator": "native",
"enable_dask": "yes",
},
{
"id": "native-nodask",
"pipeline_workflow_orchestrator": "native",
"enable_dask": "no",
},
]


def main():
"""Discover models and orchestrators, output as JSON for GitHub Actions matrix."""
import argparse

parser = argparse.ArgumentParser(description="Discover models and orchestrators for CI matrix")
parser.add_argument(
"--output",
choices=["models", "orchestrators", "both"],
default="models",
help="What to output",
)
args = parser.parse_args()

try:
if args.output == "models":
model_names = discover_model_names()
model_names = sorted(model_names)
print(json.dumps(model_names))
print(
f"# Discovered {len(model_names)} models: {', '.join(model_names)}",
file=sys.stderr,
)
elif args.output == "orchestrators":
orchestrators = get_orchestrator_configs()
print(json.dumps(orchestrators))
orchestrator_ids = [o["id"] for o in orchestrators]
print(
f"# Orchestrators: {', '.join(orchestrator_ids)}",
file=sys.stderr,
)
elif args.output == "both":
result = {
"models": sorted(discover_model_names()),
"orchestrators": get_orchestrator_configs(),
}
print(json.dumps(result))
print(
f"# Discovered {len(result['models'])} models and {len(result['orchestrators'])} orchestrator configs",
file=sys.stderr,
)

return 0
except Exception as e:
print(f"# Error discovering CI matrix config: {e}", file=sys.stderr)
# Return empty array/object on error so workflow doesn't fail
if args.output == "both":
print('{"models": [], "orchestrators": []}')
else:
print("[]")
return 1


if __name__ == "__main__":
sys.exit(main())
2 changes: 1 addition & 1 deletion .github/workflows/CI-publish.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ jobs:
- name: List files
run: ls -ratlR
- name: pypi-publish
uses: pypa/gh-action-pypi-publish@v1.9.0
uses: pypa/gh-action-pypi-publish@v1.13.0
with:
verbose: true
print-hash: true
Loading
Loading