CE: Added pages with guidelines for images on Alps

[Madeeks]

As agreed in VCUE, I have updated a set of images with foundational resources (CUDA, MPI, NCCL, NVSHMEM), deriving them from material I use myself, and demonstrated how to run them through the CE on Alps.

The intent of this material is to offer guidelines and suggestions about versions, building, and running of foundational components to use on Alps, without committing to officially supported resources.

[github-actions]

preview available: https://docs.tds.cscs.ch/272

[github-actions]

preview available: https://docs.tds.cscs.ch/272

[github-actions]

preview available: https://docs.tds.cscs.ch/272

[github-actions]

preview available: https://docs.tds.cscs.ch/272

[github-actions]

preview available: https://docs.tds.cscs.ch/272

[msimberg]

I'll try to come back to this for a proper review in the next ~week. If someone else wants to look in the mean time, please go ahead.

A question that came up while skimming through the changes is how the performance results in this PR relate to the changes in #262 and whether we should duplicate them?

Another minor note would be to add links to the "Communication libraries" sections, since they contain some useful info about environment variables etc. (sorry if I missed this and you already added them).

Finally, for the spell checker please add the remaining words to .github/actions/spelling/allow.txt (with all caps = exact match, lower case = any capitalization; I think the former is appropriate in this case).

[github-actions]

preview available: https://docs.tds.cscs.ch/272

[Madeeks]

Thanks for the tip about the spelling checker @msimberg.

Regarding references to the "Communication libraries" section, I didn't add anything at this time.
I'm absolutely open to do it, but what I would find most useful is to find an agreement between which settings should we embed directly in hooks (therefore becoming transparent to users, like a bunch of NCCL-related vars in the AWS OFI NCCL hook) and which we leave users to handle directly (e.g. in the EDF).

[msimberg]

what I would find most useful is to find an agreement between which settings should we embed directly in hooks (therefore becoming transparent to users, like a bunch of NCCL-related vars in the AWS OFI NCCL hook) and which we leave users to handle directly (e.g. in the EDF).

I think it's useful to document what we set anyway for at least two reasons:

• it serves as a useful reference to reproduce results manually and/or on other systems and makes it less "magic" (you actually see what's being set, without having to inspect the environment)
• uenv users don't have hooks to set the environment variables (there's been discussion about this, but I don't think we have a good solution yet; should they be baked into the uenv, what happens when recommendations change etc.?)

The downside is that whatever we document here may be out of sync with container engine hooks.

[RMeli]

Very useful! I left a few comments.

[RMeli]

This is bound out of date, at least temporarily. Should we provide some instructions to the user on how to retrieve this information from the container or registry instead?

[RMeli]

It is also a bit redundant with versions explicitly set below.

[RMeli]

See previous comment.

[RMeli]

See previous comment.

[RMeli]

See previous comment.

[github-actions]

preview available: https://docs.tds.cscs.ch/272

[github-actions]

preview available: https://docs.tds.cscs.ch/272

[github-actions]

preview available: https://docs.tds.cscs.ch/272

[bcumming]

These changes present some excellent information for our users (and for CSCS staff).

The docs as structured are not integrated with the rest of the documentation, namely

• the library-spcific content can move to the appropriate section of "Applications and Frameworks"
• links can be provided to existing docs (e.g. where the CXI hook is mentioned, make a link to the hook docs)

Documentation on how to compiled software, programming environments, etc are provided in the Applications & Frameworks part of the documentation https://docs.cscs.ch/software/

Like @msiberg pointed out, the logical location for this material is in the Applications&Frameworks->Communication libraries section.
Otherwise users are less likely to find these pages, and will be confused by having the same thing documentated in two locations.
For example, there is an existing MPICH page that explicitly covers how to create a container (the use case was users of the containerised CI/CD tool).
https://docs.cscs.ch/software/communication/mpich/

I think it is easiest to have a call, to speed up the process of determining what to do.