Switch deployment to mache.deploy#501
Merged
xylar merged 22 commits intoE3SM-Project:mainfrom Mar 21, 2026
Merged
Conversation
This brings in changes made on the mache side.
Collaborator
Author
TestingThis was thoroughly tested as part of #502 |
Collaborator
Author
|
Will merge this with #502 because it requires dependency updates in that branch for successful testing. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This branch replaces the legacy self-contained deployment machinery
(
configure_polaris_envs.py,deploy/bootstrap.py,deploy/shared.py,and the associated template/spec files) with the new
mache.deployframework introduced in mache 3.0.0. The result is a thin, stable
entrypoint in this repo (
deploy.py+deploy/) that delegates theheavy lifting to mache, rather than duplicating it here.
Core change:
deploy.pyand thedeploy/directoryWhat was removed
The old deployment approach lived almost entirely inside this repo:
configure_polaris_envs.pydeploy/bootstrap.pydeploy/shared.pydeploy/default.cfgdeploy/conda-dev-spec.templatedeploy/load_polaris.templatedeploy/spec-bootstrap.txtdeploy/albany_supported.txtdeploy/petsc_supported.txtdeploy/unsupported.txtIn total, roughly 2,700 lines of deployment code are removed from this
repo.
What was added
The new approach provides a lean set of files that configure and invoke
mache.deploy:deploy.pydeploy/pins.cfg, builds the CLI fromdeploy/cli_spec.json, downloadsmache/deploy/bootstrap.pyfor the pinned (or fork/branch) mache version, and delegates to itdeploy/cli_spec.json3.0.0deploy/config.yaml.j2mache deployconfig: sets project name/version, machine resolution, pixi channels, spack support flags, and hooksdeploy/hooks.pypre_pixi,pre_spack,post_deploy) that resolve the polaris version, select the correct MPI provider per machine, and locate the Spack pathdeploy/pixi.toml.j2deploy/spack.yaml.j2deploy/load.shNETCDF,NETCDFF,PNETCDF,PIO,METIS_ROOT,PARMETIS_ROOT, and build-flag env varsdeploy/pins.cfgdeploy.pyThe design means that future changes to the common deployment logic (new
machines, updated Spack recipes, pixi channel changes, etc.) only require
a mache release and a version bump in
deploy/pins.cfg, with no changesto this repo.
Supporting polaris changes
Machine configs (
polaris/machines/)environments" to reflect the new pixi-based approach.
conda-linux.cfgrenamed todefault-linux-64.cfg.conda-osx.cfgrenamed todefault-osx-64.cfg.Environment variable rename (
polaris/setup.py)The environment variable used to locate the active load script was
renamed from
LOAD_POLARIS_ENVtoPOLARIS_LOAD_SCRIPTto match theconvention used by
mache.deploy.Provenance (
polaris/provenance.py)conda listreplaced withpixi listin the provenance file written towork directories, since the environment is now managed by pixi.
pyproject.tomltermcolorruntime dependency (used by the new deploy tooling).conda*from the ruff exclusion list (no longer relevant).Documentation updates
The developer and user guide docs are updated throughout to match the new
deployment workflow:
docs/developers_guide/quick_start.md— substantially revised todescribe running
deploy.pyinstead ofconfigure_polaris_envs.py.docs/developers_guide/updating_conda.md— rewritten to coverupdating the pixi environment spec via
deploy/pixi.toml.j2anddeploy/pins.cfg.docs/developers_guide/machines/index.md— updated machinedeployment descriptions.
docs/developers_guide/updating_spack/— various pages updated toreflect the new
mache.deployspack workflow.docs/users_guide/machines/— load-script and setup instructionsupdated for all supported machines (Aurora, Chrysalis, Frontier,
Perlmutter).
tutorial docs.
Checklist
Testingcomment in the PR documents testing used to verify the changes