diff --git a/docs/archive.md b/docs/archive.md index 9fb6ef6fc45..35b9ebb99dd 100644 --- a/docs/archive.md +++ b/docs/archive.md @@ -5,6 +5,109 @@ nav_order: 15 --- Below are archived releases for RAPIDS Accelerator for Apache Spark. +## Release v26.04.0 +### Hardware Requirements: + +The plugin is designed to work on NVIDIA Volta, Turing, Ampere, Ada Lovelace, Hopper and Blackwell generation datacenter GPUs. The plugin jar is tested on the following GPUs: + + GPU Models: NVIDIA V100, T4, A10, A100, L4, H100 and B100 GPUs + +### Software Requirements: + + OS: Spark RAPIDS is compatible with any Linux distribution with glibc >= 2.28 (Please check ldd --version output). glibc 2.28 was released August 1, 2018. + Tested on Ubuntu 22.04, Ubuntu 24.04, Rocky Linux 8 and Rocky Linux 9 + + NVIDIA Driver*: R525+ + + Runtime: + Scala 2.12, 2.13 + Python, Java Virtual Machine (JVM) compatible with your spark-version. + + * Check the Spark documentation for Python and Java version compatibility with your specific + Spark version. For instance, visit `https://spark.apache.org/docs/3.4.1` for Spark 3.4.1. + + Supported Spark versions: + Apache Spark 3.3.0, 3.3.1, 3.3.2, 3.3.3, 3.3.4 + Apache Spark 3.4.0, 3.4.1, 3.4.2, 3.4.3, 3.4.4 + Apache Spark 3.5.0, 3.5.1, 3.5.2, 3.5.3, 3.5.4, 3.5.5, 3.5.6, 3.5.7, 3.5.8 + Apache Spark 4.0.0, 4.0.1, 4.0.2 + Apache Spark 4.1.1 + Scala 2.12: Spark 3.3.0 through 3.5.8 + Scala 2.13: Spark 3.5.0 through 3.5.8, and Spark 4.0.0, 4.0.1, 4.0.2, and 4.1.1 + + Supported Databricks runtime versions for Azure and AWS: + Databricks 13.3 ML LTS (GPU, Scala 2.12, Spark 3.4.1) + Databricks 14.3 ML LTS (GPU, Scala 2.12, Spark 3.5.0) + Databricks 17.3 ML LTS (GPU, Scala 2.13, Spark 4.0.0) + + Supported Dataproc versions (Debian/Ubuntu/Rocky): + GCP Dataproc 2.1 + GCP Dataproc 2.2 + GCP Dataproc 2.3 + + Supported Dataproc Serverless versions: + Spark runtime 1.1 LTS + Spark runtime 1.2 + Spark runtime 2.0 + Spark runtime 2.1 + Spark runtime 2.2 + +*Some hardware may have a minimum driver version greater than R470. Check the GPU spec sheet +for your hardware's minimum driver version. + +*For EMR support, please refer to the +[Distributions](https://docs.nvidia.com/spark-rapids/user-guide/latest/faq.html#which-distributions-are-supported) section of the FAQ. + +### RAPIDS Accelerator's Support Policy for Apache Spark +The RAPIDS Accelerator maintains support for Apache Spark versions available for download from [Apache Spark](https://spark.apache.org/downloads.html) + +### Download RAPIDS Accelerator for Apache Spark v26.04.0 + +#### CUDA 12 + +| Processor | Scala Version | Download Jar | Download Signature | Download From Maven | +|-----------|---------------|--------------|--------------------|---------------------| +| x86_64 | Scala 2.12 | [RAPIDS Accelerator v26.04.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.04.0/rapids-4-spark_2.12-26.04.0.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.04.0/rapids-4-spark_2.12-26.04.0.jar.asc) |
<dependency>| +| x86_64 | Scala 2.13 | [RAPIDS Accelerator v26.04.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/26.04.0/rapids-4-spark_2.13-26.04.0.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/26.04.0/rapids-4-spark_2.13-26.04.0.jar.asc) |
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark_2.12</artifactId>
<version>26.04.0</version>
</dependency>
<dependency>| +| arm64 | Scala 2.12 | [RAPIDS Accelerator v26.04.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.04.0/rapids-4-spark_2.12-26.04.0-cuda12-arm64.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.04.0/rapids-4-spark_2.12-26.04.0-cuda12-arm64.jar.asc) |
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark_2.13</artifactId>
<version>26.04.0</version>
</dependency>
<dependency>| +| arm64 | Scala 2.13 | [RAPIDS Accelerator v26.04.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/26.04.0/rapids-4-spark_2.13-26.04.0-cuda12-arm64.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/26.04.0/rapids-4-spark_2.13-26.04.0-cuda12-arm64.jar.asc) |
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark_2.12</artifactId>
<version>26.04.0</version>
<classifier>cuda12-arm64</classifier>
</dependency>
<dependency>| + + +#### CUDA 13 + +| Processor | Scala Version | Download Jar | Download Signature | Download From Maven | +|-----------|---------------|--------------|--------------------|---------------------| +| x86_64 | Scala 2.12 | [RAPIDS Accelerator v26.04.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.04.0/rapids-4-spark_2.12-26.04.0-cuda13.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.04.0/rapids-4-spark_2.12-26.04.0-cuda13.jar.asc) |
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark_2.13</artifactId>
<version>26.04.0</version>
<classifier>cuda12-arm64</classifier>
</dependency>
<dependency>| +| x86_64 | Scala 2.13 | [RAPIDS Accelerator v26.04.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/26.04.0/rapids-4-spark_2.13-26.04.0-cuda13.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/26.04.0/rapids-4-spark_2.13-26.04.0-cuda13.jar.asc) |
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark_2.12</artifactId>
<version>26.04.0</version>
<classifier>cuda13</classifier>
</dependency>
<dependency>| +| arm64 | Scala 2.12 | [RAPIDS Accelerator v26.04.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.04.0/rapids-4-spark_2.12-26.04.0-cuda13-arm64.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.04.0/rapids-4-spark_2.12-26.04.0-cuda13-arm64.jar.asc) |
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark_2.13</artifactId>
<version>26.04.0</version>
<classifier>cuda13</classifier>
</dependency>
<dependency>| +| arm64 | Scala 2.13 | [RAPIDS Accelerator v26.04.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/26.04.0/rapids-4-spark_2.13-26.04.0-cuda13-arm64.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/26.04.0/rapids-4-spark_2.13-26.04.0-cuda13-arm64.jar.asc) |
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark_2.12</artifactId>
<version>26.04.0</version>
<classifier>cuda13-arm64</classifier>
</dependency>
<dependency>| + + +The above packages are built against CUDA 12.9 or CUDA 13.1. They are tested on V100, T4, A10, A100, L4, H100 and GB100 GPUs. + +### Verify signature +* Download the [PUB_KEY](https://keys.openpgp.org/search?q=sw-spark@nvidia.com). +* Import the public key: `gpg --import PUB_KEY` +* Verify the signature for Scala 2.12 jar: + `gpg --verify rapids-4-spark_2.12-26.04.0.jar.asc rapids-4-spark_2.12-26.04.0.jar` +* Verify the signature for Scala 2.13 jar: + `gpg --verify rapids-4-spark_2.13-26.04.0.jar.asc rapids-4-spark_2.13-26.04.0.jar` + +The output of signature verify: + + gpg: Good signature from "NVIDIA Spark (For the signature of spark-rapids release jars)
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark_2.13</artifactId>
<version>26.04.0</version>
<classifier>cuda13-arm64</classifier>
</dependency>
<dependency>| -| x86_64 | Scala 2.13 | [RAPIDS Accelerator v26.04.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/26.04.0/rapids-4-spark_2.13-26.04.0.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/26.04.0/rapids-4-spark_2.13-26.04.0.jar.asc) |
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark_2.12</artifactId>
<version>26.04.0</version>
</dependency>
<dependency>| -| arm64 | Scala 2.12 | [RAPIDS Accelerator v26.04.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.04.0/rapids-4-spark_2.12-26.04.0-cuda12-arm64.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.04.0/rapids-4-spark_2.12-26.04.0-cuda12-arm64.jar.asc) |
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark_2.13</artifactId>
<version>26.04.0</version>
</dependency>
<dependency>| -| arm64 | Scala 2.13 | [RAPIDS Accelerator v26.04.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/26.04.0/rapids-4-spark_2.13-26.04.0-cuda12-arm64.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/26.04.0/rapids-4-spark_2.13-26.04.0-cuda12-arm64.jar.asc) |
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark_2.12</artifactId>
<version>26.04.0</version>
<classifier>cuda12-arm64</classifier>
</dependency>
<dependency>| +| x86_64 | Scala 2.12 | [RAPIDS Accelerator v26.04.1](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.04.1/rapids-4-spark_2.12-26.04.1.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.04.1/rapids-4-spark_2.12-26.04.1.jar.asc) |
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark_2.13</artifactId>
<version>26.04.0</version>
<classifier>cuda12-arm64</classifier>
</dependency>
<dependency>| +| x86_64 | Scala 2.13 | [RAPIDS Accelerator v26.04.1](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/26.04.1/rapids-4-spark_2.13-26.04.1.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/26.04.1/rapids-4-spark_2.13-26.04.1.jar.asc) |
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark_2.12</artifactId>
<version>26.04.1</version>
</dependency>
<dependency>| +| arm64 | Scala 2.12 | [RAPIDS Accelerator v26.04.1](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.04.1/rapids-4-spark_2.12-26.04.1-cuda12-arm64.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.04.1/rapids-4-spark_2.12-26.04.1-cuda12-arm64.jar.asc) |
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark_2.13</artifactId>
<version>26.04.1</version>
</dependency>
<dependency>| +| arm64 | Scala 2.13 | [RAPIDS Accelerator v26.04.1](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/26.04.1/rapids-4-spark_2.13-26.04.1-cuda12-arm64.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/26.04.1/rapids-4-spark_2.13-26.04.1-cuda12-arm64.jar.asc) |
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark_2.12</artifactId>
<version>26.04.1</version>
<classifier>cuda12-arm64</classifier>
</dependency>
<dependency>| #### CUDA 13 | Processor | Scala Version | Download Jar | Download Signature | Download From Maven | |-----------|---------------|--------------|--------------------|---------------------| -| x86_64 | Scala 2.12 | [RAPIDS Accelerator v26.04.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.04.0/rapids-4-spark_2.12-26.04.0-cuda13.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.04.0/rapids-4-spark_2.12-26.04.0-cuda13.jar.asc) |
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark_2.13</artifactId>
<version>26.04.1</version>
<classifier>cuda12-arm64</classifier>
</dependency>
<dependency>| -| x86_64 | Scala 2.13 | [RAPIDS Accelerator v26.04.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/26.04.0/rapids-4-spark_2.13-26.04.0-cuda13.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/26.04.0/rapids-4-spark_2.13-26.04.0-cuda13.jar.asc) |
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark_2.12</artifactId>
<version>26.04.0</version>
<classifier>cuda13</classifier>
</dependency>
<dependency>| -| arm64 | Scala 2.12 | [RAPIDS Accelerator v26.04.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.04.0/rapids-4-spark_2.12-26.04.0-cuda13-arm64.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.04.0/rapids-4-spark_2.12-26.04.0-cuda13-arm64.jar.asc) |
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark_2.13</artifactId>
<version>26.04.0</version>
<classifier>cuda13</classifier>
</dependency>
<dependency>| -| arm64 | Scala 2.13 | [RAPIDS Accelerator v26.04.0](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/26.04.0/rapids-4-spark_2.13-26.04.0-cuda13-arm64.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/26.04.0/rapids-4-spark_2.13-26.04.0-cuda13-arm64.jar.asc) |
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark_2.12</artifactId>
<version>26.04.0</version>
<classifier>cuda13-arm64</classifier>
</dependency>
<dependency>| +| x86_64 | Scala 2.12 | [RAPIDS Accelerator v26.04.1](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.04.1/rapids-4-spark_2.12-26.04.1-cuda13.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.04.1/rapids-4-spark_2.12-26.04.1-cuda13.jar.asc) |
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark_2.13</artifactId>
<version>26.04.0</version>
<classifier>cuda13-arm64</classifier>
</dependency>
<dependency>| +| x86_64 | Scala 2.13 | [RAPIDS Accelerator v26.04.1](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/26.04.1/rapids-4-spark_2.13-26.04.1-cuda13.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/26.04.1/rapids-4-spark_2.13-26.04.1-cuda13.jar.asc) |
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark_2.12</artifactId>
<version>26.04.1</version>
<classifier>cuda13</classifier>
</dependency>
<dependency>| +| arm64 | Scala 2.12 | [RAPIDS Accelerator v26.04.1](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.04.1/rapids-4-spark_2.12-26.04.1-cuda13-arm64.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.04.1/rapids-4-spark_2.12-26.04.1-cuda13-arm64.jar.asc) |
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark_2.13</artifactId>
<version>26.04.1</version>
<classifier>cuda13</classifier>
</dependency>
<dependency>| +| arm64 | Scala 2.13 | [RAPIDS Accelerator v26.04.1](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/26.04.1/rapids-4-spark_2.13-26.04.1-cuda13-arm64.jar) | [Signature](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/26.04.1/rapids-4-spark_2.13-26.04.1-cuda13-arm64.jar.asc) |
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark_2.12</artifactId>
<version>26.04.1</version>
<classifier>cuda13-arm64</classifier>
</dependency>
<dependency>| The above packages are built against CUDA 12.9 or CUDA 13.1. They are tested on V100, T4, A10, A100, L4, H100 and GB100 GPUs. @@ -103,15 +103,21 @@ The above packages are built against CUDA 12.9 or CUDA 13.1. They are tested on * Download the [PUB_KEY](https://keys.openpgp.org/search?q=sw-spark@nvidia.com). * Import the public key: `gpg --import PUB_KEY` * Verify the signature for Scala 2.12 jar: - `gpg --verify rapids-4-spark_2.12-26.04.0.jar.asc rapids-4-spark_2.12-26.04.0.jar` + `gpg --verify rapids-4-spark_2.12-26.04.1.jar.asc rapids-4-spark_2.12-26.04.1.jar` * Verify the signature for Scala 2.13 jar: - `gpg --verify rapids-4-spark_2.13-26.04.0.jar.asc rapids-4-spark_2.13-26.04.0.jar` + `gpg --verify rapids-4-spark_2.13-26.04.1.jar.asc rapids-4-spark_2.13-26.04.1.jar` The output of signature verify: gpg: Good signature from "NVIDIA Spark (For the signature of spark-rapids release jars)
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark_2.13</artifactId>
<version>26.04.1</version>
<classifier>cuda13-arm64</classifier>
</dependency>