Exchange state halos at every Runge-Kutta stage

[mwarusz]

#354 added a halo exchange for tracers at every Runge-Kutta stage. To have valid halo data for the del4 velocity tendency, we also need to perform it at every stage for the state variables.

Fixes #376.

Checklist

• Linting
  • Pre-commit run locally
• Building
  • CMake build does not produce any new warnings from changes in this PR
• Testing
  • Add a comment to the PR titled Testing with the following:
    • Which machines CTest unit tests
      have been run on and indicate that are all passing.
    • The Polaris omega_pr test suite
      has passed, using the Polaris e3sm_submodules/Omega baseline
    • Document machine(s), compiler(s), and the build path(s) used for -p for both the baseline (Polaris e3sm_submodules/Omega) and the PR build
    • Indicate "All tests passed" or document failing tests
    • Document testing used to verify the changes including any tests that are added/modified/impacted.

[mwarusz]

Testing

CTest unit tests

• Machine: aurora
• Compiler: oneapi-ifx
• Build type: Release
• Result: All 39 tests passed
• Log: /lus/flare/projects/E3SM_Dec/mwaruszewski/omega-pr-testing/rk-state-exchng/build-ifx/Testing/Temporary/LastTest.log

• Machine: frontier
• Compiler: craygnu-mphipcc
• Build type: Release
• Result: All 39 tests passed
• Log: /lustre/orion/cli115/scratch/mwaruszewski/omega-pr-testing/rk-state-exchng/build-craygnu-mphipcc/Testing/Temporary/LastTest.log

Polaris omega_pr suite

• Baseline workdir: /lus/flare/projects/E3SM_Dec/mwaruszewski/omega-pr-testing/rk-state-exchng/baseline-ifx/
• Baseline build: /lus/flare/projects/E3SM_Dec/mwaruszewski/omega-pr-testing/rk-state-exchng/baseline-ifx/build
• PR build: /lus/flare/projects/E3SM_Dec/mwaruszewski/omega-pr-testing/rk-state-exchng/build-ifx
• PR workdir: /lus/flare/projects/E3SM_Dec/mwaruszewski/omega-pr-testing/rk-state-exchng/pr-ifx
• Machine: aurora
• Compiler: oneapi-ifx
• Build type: <Debug|Release>
• Log: /lus/flare/projects/E3SM_Dec/mwaruszewski/omega-pr-testing/rk-state-exchng/pr-ifx/polaris_omega_pr.o8401396
• Result:
  • Diffs (1 of 9):
    • ocean/planar/manufactured_solution/convergence_both/del4

Additional tests

On aurora I ran the ocean/planar/manufactured_solution/convergence_both/del4 three times, each with different tasks configuration. The first run was used as a baseline for the second and third. I did this using the polaris Omega submodule and the PR branch. As expected, with the polaris Omega submodule there were baseline failures. On the PR branch there were none. Logs:

polaris_run_1: /lus/flare/projects/E3SM_Dec/mwaruszewski/omega-pr-testing/rk-state-exchng/mf-del4-polaris-default/polaris_custom.o8401536
polaris_run_2: /lus/flare/projects/E3SM_Dec/mwaruszewski/omega-pr-testing/rk-state-exchng/mf-del4-polaris-goal300/polaris_custom.o8401546
polaris_run_3: /lus/flare/projects/E3SM_Dec/mwaruszewski/omega-pr-testing/rk-state-exchng/mf-del4-polaris-goal400/polaris_custom.o8401560

PR_run_1: /lus/flare/projects/E3SM_Dec/mwaruszewski/omega-pr-testing/rk-state-exchng/mf-del4-xchng-default/polaris_custom.o8401587
PR_run_2: /lus/flare/projects/E3SM_Dec/mwaruszewski/omega-pr-testing/rk-state-exchng/mf-del4-xchng-goal300/polaris_custom.o8401604
PR_run_3: /lus/flare/projects/E3SM_Dec/mwaruszewski/omega-pr-testing/rk-state-exchng/mf-del4-xchng-goal400/polaris_custom.o8401638

[Copilot]

Pull request overview

Ensures state-variable halo data is valid at every Runge–Kutta stage so velocity
hyper-diffusion (del4) tendencies are consistent across MPI decompositions,
fixing #376.

Changes:

• RK4: exchange provisional state halos at every stage (not just a single
  stage).
• RK2: exchange midpoint state halos before evaluating midpoint tendencies.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
components/omega/src/timeStepping/RungeKutta4Stepper.cpp	Exchanges provisional state halos at each RK stage before computing tendencies.
components/omega/src/timeStepping/RungeKutta2Stepper.cpp	Exchanges midpoint state halos prior to midpoint tendency evaluation.

[brian-oneill]

Looks good, ran the ctests successfully on pm-cpu and frontier CPU & GPU. Approved based on inspection and @mwarusz's polaris testing above.