Pr 780

[NickGeneva]

Earth2Studio Pull Request

Description

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.
• The CHANGELOG.md is up to date with these changes.
• An issue is linked to this pull request.
• Assess and address Greptile feedback (AI code review bot for guidance; use discretion, addressing all feedback is not required).

Dependencies

[NickGeneva]

/blossom-ci

[NickGeneva]

/blossom-ci

[greptile-apps]

Greptile Summary

This PR introduces CAMS_FX, a new ForecastSource for the Copernicus Atmosphere Monitoring Service (CAMS) Global Atmospheric Composition Forecasts dataset, along with its companion CAMSGlobalLexicon. It also registers new air-quality vocabulary entries in E2STUDIO_VOCAB and extends the documentation badge filters with region:europe and product:airquality.

Key changes:

• earth2studio/data/cams.py – New CAMS_FX class supporting surface and pressure-level variable retrieval via the ADS cdsapi, with local caching, async support, and separate request batching for surface vs. pressure-level variables.
• earth2studio/lexicon/cams.py – New CAMSGlobalLexicon mapping e2s variable names to CAMS API names and NetCDF keys for surface meteorology, aerosol optical depth, trace gases, and pressure-level fields.
• earth2studio/lexicon/base.py – Ten new CAMS-specific entries (AOD and trace-gas columns) added to E2STUDIO_VOCAB.
• Tests – Comprehensive mock-based and integration tests for both the data source and lexicon; test_cds.py gains an autouse fixture to pin the CDSAPI_URL environment variable.
• Docs – Badge filters updated and data.CAMS_FX added to the forecast sources toctree.

The primary concern is that _validate_time does not enforce the CAMS-specific constraint that forecasts are only initialized at 00:00 and 12:00 UTC — any other hour will produce an API error at runtime.

Confidence Score: 3/5

The PR is mostly safe but contains a logic gap in time validation that will cause runtime API failures for users who supply non-00:00/12:00 initialization times.

The implementation is well-structured and tested, but _validate_time does not check that the initialization hour is 0 or 12 — a hard constraint imposed by the CAMS API. Any call with an unsupported hour (e.g. 06:00 UTC) will fail at the remote API level with a potentially confusing error rather than a clear local validation message. The missing product:airquality badge in the class docstring is a minor documentation inconsistency. All other logic, caching, and lexicon entries look correct.

earth2studio/data/cams.py — specifically the _validate_time class method

Important Files Changed

Filename	Overview
earth2studio/data/cams.py	New CAMS_FX forecast data source; missing initialization-hour validation (only 00:00 and 12:00 UTC are valid CAMS init times) and product:airquality badge in class docstring
earth2studio/lexicon/cams.py	New CAMSGlobalLexicon mapping e2s variable names to CAMS API names and NetCDF keys; vocabulary format is consistent and the identity modifier pattern matches other lexicons
earth2studio/lexicon/base.py	Adds 10 new CAMS-specific variable entries (aerosol AOD and trace gas columns) to E2STUDIO_VOCAB; additions look correct and consistent
test/data/test_cams.py	Comprehensive unit and mock tests for CAMS_FX; does not include a test for invalid initialization hours (e.g. 06:00 UTC), which mirrors the missing validation in the source
test/lexicon/test_cams_lexicon.py	Well-structured lexicon tests covering surface, AOD, trace-gas, and pressure-level variables; also validates VOCAB entry format
test/data/test_cds.py	Adds a monkeypatched CDSAPI_URL fixture so CDS tests target the correct endpoint; straightforward improvement
earth2studio/data/init.py	Exports CAMS_FX from the data package; change is minimal and correct
earth2studio/lexicon/init.py	Exports CAMSGlobalLexicon from the lexicon package; change is minimal and correct
docs/modules/datasources_analysis.rst	Adds region:europe and product:airquality to the badge filter; consistent with the new CAMS data source
docs/modules/datasources_forecast.rst	Adds region:europe and product:airquality badge filters and registers CAMS_FX in the forecast source toctree

Comments Outside Diff (1)

1. earth2studio/data/cams.py, line 514-521 (link)

   Fixed temp-cache path may conflict across concurrent instances

   When cache=False, the cache property returns the fixed path …/cams/tmp_cams_fx. If two CAMS_FX(cache=False) instances run concurrently (e.g. in a multi-process workflow), one instance's shutil.rmtree(self.cache) call in __call__ will delete files that the other instance may still be using. The CDS data source in this repo uses the same pattern, but it is worth noting for users running parallel pipelines. A per-instance UUID suffix (e.g. tempfile.mkdtemp) would make the temp directory unique.

_{Reviews (1): Last reviewed commit: "Merge branch 'main' into pr-780" | Re-trigger Greptile}

[greptile-apps]

Missing CAMS initialization-hour validation

CAMS Global Atmospheric Composition Forecasts are only produced at 00:00 UTC and 12:00 UTC. The current _validate_time only enforces that minutes and seconds are zero — it accepts any hour (e.g. 06:00, 18:00). Passing a non-supported hour will produce an API error or silently return unexpected data.

A check for the valid initialization hours should be added alongside the existing minute/second check:

Suggested change

	"are only available at 3-hourly intervals (0, 3, 6, ...)."
	)
	# Download surface variables
	surface_ds: xr.Dataset | None = None
	if surface_infos:
	surface_api_vars = list(dict.fromkeys(vi.api_name for vi in surface_infos))
	nc_path = self._download_cached(time, surface_api_vars, lead_hours)
	surface_ds = xr.open_dataset(nc_path, decode_timedelta=False)
	# Download pressure-level variables (grouped by unique levels)
	pressure_ds: xr.Dataset | None = None
	if pressure_infos:
	@classmethod
	def _validate_time(cls, times: list[datetime]) -> None:
	"""Validate that requested times are valid for CAMS Global forecast."""
	for t in times:
	t_naive = t.replace(tzinfo=None) if t.tzinfo else t
	if t_naive < cls.CAMS_MIN_TIME:
	raise ValueError(
	f"Requested time {t} is before CAMS Global forecast availability "
	f"(earliest: {cls.CAMS_MIN_TIME})"
	)
	if t_naive.minute != 0 or t_naive.second != 0:
	raise ValueError(
	f"Requested time {t} must be on the hour for CAMS Global forecast"
	)
	if t_naive.hour not in (0, 12):
	raise ValueError(
	f"Requested time {t} has unsupported initialization hour "
	f"({t_naive.hour:02d}:00 UTC). CAMS Global forecasts are only "
	"available at 00:00 and 12:00 UTC."
	)

[greptile-apps]

Missing product:airquality badge in class docstring

region:europe and product:airquality were added to the RST badge-filter directives, yet the CAMS_FX class docstring only carries product:wind product:temp product:atmos. The class also exposes aerosol optical depth and trace-gas variables, so product:airquality (and possibly region:europe) should be listed here to ensure the class is surfaced correctly in the documentation filter.

Suggested change

	data_arrays.append(da)
	if not self._cache:
	shutil.rmtree(self.cache)
	Badges
	------
	region:global region:europe dataclass:simulation product:wind product:temp product:atmos product:airquality

[greptile-apps]

Silent fallback isel[d] = 0 for unexpected dimensions

When a dimension is neither forecast_period nor a pressure-level dimension, the code silently selects index 0 with no warning:

else:
    isel[d] = 0

If a future CAMS NetCDF adds an unexpected extra dimension (e.g. ensemble member, realisation, step), the code will silently extract only the first slice without alerting the user. Consider emitting a logger.warning in this branch so unexpected dimensions are visible:

else:
    logger.warning(
        f"Unexpected dimension '{d}' in field '{nc_key}'; selecting index 0."
    )
    isel[d] = 0

[NickGeneva]

/blossom-ci