Conditional auxiliary halo exchange when turbulence is not used

[MikeW097]

Following Harald’s suggestion:

“With respect to the deactivation of the aux field, @KannanMasilamani said that we could do that if we then compute derived quantities on the fly for tracking. (Introducing a flag on whether we have the aux field available or not). If you already have a change where you deactivate the aux field machinery, you could put that on a branch and open a PR for that, so that this could be implemented on the branch.”

This PR implements the first step in that direction by avoiding the aux-field halo exchange when the turbulence model is not required.

[haraldkl]

@MikeW097 thanks for opening this branch for work on this.

I think we should introduce a dedicated configuration option to explicitly turn auxfield computation on and off, and make that one required to be on if there is turbulence to be computed. What we then need in addition is to respect this flag in the computation of derived quantities for tracking (as we can take the already computed values from the auxfield if available, but otherwise have to compute them).

[MikeW097]

@haraldkl I understand the point about tracking derived quantities, but I think handling this implicitly based on the turbulence model solves everything without the need for an extra configuration option.

Based on my understand:

• When turbulence is ON: The gradient is needed in the equilibrium/solving process. Therefore, the auxiliary communication is absolutely necessary.
• When turbulence is OFF: The gradient is not needed in the solving process. The auxiliary communication is unnecessary and just adds overheads.

Adding an implicit control is enough in both auxiliary and tracking:

• If Turbulence is ON: Aux comm is active globally. Tracking reads the derived quantities.
• If Turbulence is OFF: Aux comm is inactive globally. Tracking computes derived quantities on the fly if the user requests them.

[haraldkl]

@MikeW097 sounds reasonable, we can introduce the flag, and set it purely based on solver logic, without the possibility to change it in the user configuration.

[haraldkl]

I think, we can push this into scheme%header instead (see 2973f51), but maybe we could deactivate the auxfield altogether when this flag happens to be true? Then I'd add an active or not flag in the auxfields themselves and we do not need to extend the interfaces, as that object gets passed in already.

[MikeW097]

@haraldkl It would be possible to fully deactivate the auxfield computation, but that would require more intrusive changes. Several kernels at the moment directly rely on density and velocity stored in the auxfield. Completely skipping aux computation would therefore require restructuring the kernels so that the physical variables are computed on the fly inside the kernel itself.
For a minor change, maybe it is more contained to only deactivate the aux halo communication when it is not needed, while keeping the auxfield computation itself.

[haraldkl]

@MikeW097 OK, thanks, we could still consider pushing the flag into the auxfield type itself during initialization. Then we do not need to carry it around in all the calls. Otherwise, we can stick with it as is and merge this.

[MikeW097]

@haraldkl Thank you for your feedback. I agree that it’s best to keep the interface unchanged, and it can also avoid any unintended downstream effects inside ApesMate.

As my recent changes were on top of my previous commit rather than yours. To avoid overwriting your work, I’ve pushed my updates to a new branch instead of force-updating the existing one. Please see #55 for new changes.

[haraldkl]

@MikeW097 as far as I can see, we would only merge #55, and there then is no need for this MR, so we would close it, right?

[MikeW097]

@haraldkl Yes, and also apes-suite/musubi#58, if you prefers the implementation shown in #55.

[haraldkl]

I think #55 would be preferable, as we keep the flag in the entity that is directly concerned with it.