Add build-ci for spack-config

[CodeGat]

Closes #86

Background

spack-config is the final piece of the puzzle to be tested by build-ci.

This PR takes inspiration from https://github.com/ACCESS-NRI/access-spack-packages/blob/main/.github/workflows/ci.yml for the general structure of the workflow.

Essentially, we allow two triggers: pull_request and workflow_dispatch, which function in the following ways:

pull_request

When a PR to spack-config is made, if there are changes to the common* directories (which are proper files symlinked to in the v* directories), we find all the v*/ci-* directories that have files that symlink to the changed files. This is our set of versions to test the common* changes against - using spack manifests that we source from this repository.

Note

Since this is using build-ci (which has no connection to supercomputers) we can only test ci-upstream/ci-runner spack configurations - not gadi or setonix

If a spack configuration to be tested is for the ci-upstream, we use GitHub-hosted runners, otherwise we use self-hosted, as upstream images don't require the custom volumes that the runner images do.

workflow_dispatch

Via the Web UI, one can choose the versions of spack/spack-config/builtin-spack-packages/access-spack-packages, as well as a list of configuration directories such as v1.1/ci-runner, and run the tests. This can be good for testing specific combinations.

The PR

• Add a new workflow, ci.yml, that runs on PRs or through the Web UI

Testing

PR Trigger

Tested via #96 - works as expected. Note, the errors are because of ACCESS-NRI/build-ci#268, which will be addressed separately.

Limitations

• At the moment, since we only look for changed files in common*, symlinks that have been added to a configuration directory won't be found - a change to common* must happen first. Potentially solved in b9ec8e0
• There is potential for some duplication across manifests (see Investigation: Centralize build-ci Manifests build-ci#264) so we need to decide where we source our manifests We have the ability to source manifests from other repositories now, but we will start by sourcing manifests locally.

[CodeGat]

Consider a files_yaml approach to the tj-actions/changed-files-action, in which we have files changed in common* and separately changed files in v*, since the latter won't have to have the "find config directories based on symlinked changed files" bit. This would solve the first limitation.

EDIT: Done in b9ec8e0

[CodeGat]

The final question is what manifests do we test for spack-config (a question that I hope to get @harshula s opinion on)?

I see there being a few options, from least to most coverage:

• A representative subset of manifests across all our models/components, stored locally in spack-config. These can go out of sync with MCRs/access-spack-packages as they are only a representative subset to make sure nothing is super broken regarding configuration.

• A representative subset of manifests from access-spack-packages. These are synced up with changes from access-spack-packages, through the proposed inputs.spack-manifest-repository for build-ci (see ACCESS-NRI/build-ci#265)

• A representative subset of manifests from MCRs. These will be synced up with changes from their respective MCRs, through the proposed inputs.spack-manifest-repository for build-ci.

• All manifests defined in access-spack-packages. These are synced up with changes from access-spack-packages, as above. This could potentially take a long time, especially if changes in common* affect multiple major versions of spack.

• All manifests defined in their respective MCRs. These are synced up with changes from their respective MCRs, as above. This could potentially take a REALLY long time, especially if changes in common* affect multiple major versions of spack, but we would be certain that all the components would work, based on their own tests.

[CodeGat]

After a conversation with @aidanheerdegen - we'll go with the simplest for now (a representative set of manifests stored in spack-config).
Possibly the workflow_dispatch could have larger subsets of manifests from access-spack-packages?

[CodeGat]

This is suitably tested, but will remain drafted until ACCESS-NRI/build-ci#268 is solved, just in case it requires a rework of the GitHub-hosted part of this workflow.