Sync the pre-commit checks with what CI is doing

[loverdos]

Description and remarks

• Run the pre-commit hook in CI. It basically supersedes most other tests, hence some deletions of .github/workflows/sdk.yml etc.
• Run times are from 6-8 minutes to 24-26 minutes depending on a) if the cache kicks in, b) if GitHub is having a good day and does not fail cache saving.
• When the cache works, building and testing everything is faster than selectively building/testing individual crates, not to mention that we always get the dependencies right.
• Add one pre-commit action to handle a CLI integration test (for CLI completions in bash)
• Add one pre-commit check for executable scripts. We are now being strict, in the sense if something looks like a shell script, it must be executable.
• Introduce a two-layered CI cache for a) the main branch, b) PRs. Both interoperate.
• Unfortunately, the caching system of GitHub is unreliable. Sometimes caches are saved, sometimes they are not. Still, anecdotally we are doing better than the current (previous to this PR) setup, where we observe that generally cache does not work the way we use it.
• I have come to the conclusion that it's better to use actions/cache manually, rather than relying on marketplace GitHub actions, such as actions-rust-lang/setup-rust-toolchain or Swatinem/rust-cache. Generally, GitHub actions are not composable.
• At some point, when everything seemed stable, I introduced sccache. It looked promising, until I started getting weird errors out of the blue, like warning: ring@0.17.14: sccache: error: failed to create directory ~/.cache/sccache/preprocessor, and so I abandoned sccache altogether. Life is short.
• In general, though this was an interesting learning exercise, it has mostly been full of frustrations. I am certain I did not get everything right and there is room for improvement, but I prefer we move to another system/way of doing CI than spend time fighting GitHub actions. Honestly, good old Jenkins on a vanilla VM would be better/more reliable (but I am not saying we should do that).
• I tried several variations of making reusable GitHub actions, but I decided to stop here. Please do not recommend more in your review. Feel free to open a ticket to address this in the future.
• There are some other workflows that overlap, e.g. checking for typos. TODO Improve quality without remorse (nexus-sdkv0.3.0) #272
• All of these notes will go to the commit message.

References

• Sync the pre-commit checks with what CI is doing. #257

Checklist

• keep a changelog
• write tests
• create tickets for TODO: <> statements
• create or update documentation

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[zanicar]

LGTM. Although someone with more CI experience may want to confirm the details.

[zanicar]

LGTM