Upgrade to async-capable (for kernel launches and memory transfers) rust-cuda (#276)

* Initial progress towards using rust-cuda with async

Some progress

Further async integration progress, rustcoalescence fails to compile

Some progress with dispatch coersion

Small cleanup

Cleanup cuda algorithm coersion

Some more cleanup

Add back missing Backup for SeaHash and WyHash rngs

Fix CUDA kernel extraneous pub exports

Minor improvement of the event buffer hack

Remove unused control_flow_enum feature

Revert Copy for [Indexed]Location

Revert new clone

Update to rust-cuda with async kernel launch async return

Update to latest rust-cuda

Fix rustfmt

Temporary fix to allow CUDA algorithm linking

Small cleanup, mostly of unused clippy allows

Small improvement to CUDA EventBuffer

Try trait-based kernel signature check

Update rust-toolchain

Fix clippy lints

Try with const match instead

Try with memcmp intrinsic

Try out experimental const-type-layout with compression

Try interning all const layout strings

Try check

Try check again

* Small fixes after rebase

* Use cuda with (hopefully) much faster type checks

* Try with llvm-bitcode-linker

* Upgrade to latest ptx-builder

* Fix codecov action

* Fix RUSTFLAGS merging for ptx kernels in codecov CI

https://blog.rust-lang.org/inside-rust/2023/08/24/cargo-config-merging.html
https://doc.rust-lang.org/cargo/reference/config.html#command-line-overrides

* Update const-type-layout

Author: juntyr

.github/workflows/ci.yml

@@ -32,13 +32,6 @@ jobs:
         run: |
           sudo apt-get update -q
           sudo apt-get install libopenmpi-dev -y --no-install-recommends
-
-      - name: Install the rust-ptx-linker
-        run: |
-          wget https://apt.llvm.org/llvm.sh && chmod +x llvm.sh
-          sudo ./llvm.sh $(rustc --version -v | grep -oP "LLVM version: \K\d+")
-          rm llvm.sh
-          cargo install rust-ptx-linker --git https://github.com/juntyr/rust-ptx-linker --force
       
       - name: Check the default features
         run: cargo check
@@ -64,13 +57,6 @@ jobs:
         run: |
           sudo apt-get update -q
           sudo apt-get install libopenmpi-dev -y --no-install-recommends
-
-      - name: Install the rust-ptx-linker
-        run: |
-          wget https://apt.llvm.org/llvm.sh && chmod +x llvm.sh
-          sudo ./llvm.sh $(rustc --version -v | grep -oP "LLVM version: \K\d+")
-          rm llvm.sh
-          cargo install rust-ptx-linker --git https://github.com/juntyr/rust-ptx-linker --force
       
       - name: Run the test-suite
         run: cargo test --workspace --no-fail-fast
@@ -104,13 +90,6 @@ jobs:
           sudo apt-get update -q
           sudo apt-get install libopenmpi-dev -y --no-install-recommends
 
-      - name: Install the rust-ptx-linker
-        run: |
-          wget https://apt.llvm.org/llvm.sh && chmod +x llvm.sh
-          sudo ./llvm.sh $(rustc --version -v | grep -oP "LLVM version: \K\d+")
-          rm llvm.sh
-          cargo install rust-ptx-linker --git https://github.com/juntyr/rust-ptx-linker --force
-
       - name: Check the code style for the default features
         run: cargo clippy -- -D warnings
 

.github/workflows/coverage.yml

@@ -31,26 +31,26 @@ jobs:
           sudo apt-get update -q
           sudo apt-get install libopenmpi-dev -y --no-install-recommends
 
-      - name: Install the Rust toolchain
+      - name: Generate the coverage data
         run: |
-          cargo install grcov --force --locked
-          rustup component add llvm-tools-preview
-
-      - name: Install the rust-ptx-linker
+          cargo clean
+          cargo \
+            --config "target.'cfg(all())'.rustflags=['-Cinstrument-coverage']" \
+            test --workspace --all-targets
+        env:
+          CARGO_INCREMENTAL: 0
+          RUSTDOCFLAGS: -Cinstrument-coverage
+          LLVM_PROFILE_FILE: coverage/coverage-%p-%m.profraw
+      
+      - name: Download grcov
         run: |
-          wget https://apt.llvm.org/llvm.sh && chmod +x llvm.sh
-          sudo ./llvm.sh $(rustc --version -v | grep -oP "LLVM version: \K\d+")
-          rm llvm.sh
-          cargo install rust-ptx-linker --git https://github.com/juntyr/rust-ptx-linker --force
-
-      - name: Generate the code coverage
+          curl -sL https://github.com/mozilla/grcov/releases/download/v0.8.18/grcov-x86_64-unknown-linux-gnu.tar.bz2 | tar jxf -
+          chmod +x ./grcov
+      
+      - name: Generate the coverage reports
         run: |
-          RUSTFLAGS="-Cinstrument-coverage" \
-          LLVM_PROFILE_FILE="codecov-%p-%m.profraw" \
-          cargo test --workspace
-
-          grcov . -s . --binary-path ./target/debug/ \
-            -t cobertura -o cobertura.xml --branch \
+          ./grcov . -s . --binary-path ./target/debug/deps \
+            -t lcov -o coverage.lcov --branch \
             --keep-only "necsim/*" \
             --keep-only "rustcoalescence/*" \
             --ignore-not-existing \
@@ -62,4 +62,5 @@ jobs:
         uses: codecov/codecov-action@v1
         with:
           token: ${{ secrets.CODECOV_TOKEN }}
+          files: coverage.lcov
           fail_ci_if_error: true

.github/workflows/rustdoc.yml

@@ -21,13 +21,6 @@ jobs:
           sudo apt-get update -q
           sudo apt-get install libopenmpi-dev -y --no-install-recommends
 
-      - name: Install the rust-ptx-linker
-        run: |
-          wget https://apt.llvm.org/llvm.sh && chmod +x llvm.sh
-          sudo ./llvm.sh $(rustc --version -v | grep -oP "LLVM version: \K\d+")
-          rm llvm.sh
-          cargo install rust-ptx-linker --git https://github.com/juntyr/rust-ptx-linker --force
-
       - name: Build the Documentation
         run: |
           RUSTDOCFLAGS="\

.gitpod.Dockerfile

@@ -8,15 +8,14 @@ RUN echo "debconf debconf/frontend select Noninteractive" | sudo debconf-set-sel
     echo "keyboard-configuration keyboard-configuration/layout select 'English (US)'" | sudo debconf-set-selections && \
     echo "keyboard-configuration keyboard-configuration/layoutcode select 'us'" | sudo debconf-set-selections && \
     echo "resolvconf resolvconf/linkify-resolvconf boolean false" | sudo debconf-set-selections && \
-    wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-ubuntu1804.pin && \
-    sudo mv cuda-ubuntu1804.pin /etc/apt/preferences.d/cuda-repository-pin-600 && \
-    sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub && \
-    sudo add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/ /" && \
+    wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.0-1_all.deb -O cuda_keyring.deb && \
+    sudo dpkg -i cuda_keyring.deb && \
+    rm cuda_keyring.deb && \
+    wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-ubuntu2204.pin && \
+    sudo mv cuda-ubuntu2204.pin /etc/apt/preferences.d/cuda-repository-pin-600 && \
+    sudo add-apt-repository deb http://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/ / && \
     sudo apt-get update -q && \
-    sudo apt-get install cuda -y --no-install-recommends && \
-    wget https://apt.llvm.org/llvm.sh && chmod +x llvm.sh && \
-    sudo ./llvm.sh $(rustc --version -v | grep -oP "LLVM version: \K\d+") && \
-    rm llvm.sh && \
+    sudo apt-get install cuda-12-3 -y --no-install-recommends && \
     sudo apt-get clean autoclean && \
     sudo apt-get autoremove -y && \
     sudo rm -rf /var/lib/{apt,dpkg,cache,log}/
@@ -31,6 +30,5 @@ RUN sudo apt-get update -q && \
     sudo apt-get autoremove -y && \
     sudo rm -rf /var/lib/{apt,dpkg,cache,log}/
 
-RUN cargo install rust-ptx-linker --git https://github.com/juntyr/rust-ptx-linker --force && \
-    cargo install cargo-mpirun --force && \
+RUN cargo install cargo-mpirun --force && \
     cargo install cargo-reaper --git https://github.com/juntyr/grim-reaper --force