From 02df101d8ca0165aaf5a2d62dc49607ca21903b3 Mon Sep 17 00:00:00 2001 From: astephenson Date: Sun, 29 Jun 2025 21:44:46 -0700 Subject: [PATCH 1/3] Updated README dependency instructions for flash-attn Updated ReadME to correct the flash-attn==2.7.4.post1 in order to prevent ABI mismatches with new flash-attn versions. --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 538a9248f..e7b083d4e 100644 --- a/README.md +++ b/README.md @@ -66,11 +66,11 @@ uv venv openr1 --python 3.11 && source openr1/bin/activate && uv pip install --u > [!TIP] > For Hugging Face cluster users, add `export UV_LINK_MODE=copy` to your `.bashrc` to suppress cache warnings from `uv` -Next, install vLLM and FlashAttention: +Next, install vLLM and FlashAttention (use version 2.7.4.post1 to avoid ABI mismatches): ```shell uv pip install vllm==0.8.5.post1 -uv pip install setuptools && uv pip install flash-attn --no-build-isolation +uv pip install setuptools && uv pip install flash-attn==2.7.4.post1 --no-build-isolation ``` This will also install PyTorch `v2.6.0` and it is **very important** to use this version since the vLLM binaries are compiled for it. You can then install the remaining dependencies for your specific use case via `pip install -e .[LIST OF MODES]`. For most contributors, we recommend: From 16ad47c7ca2c318f47a92bd4ab031ad2531c6c26 Mon Sep 17 00:00:00 2001 From: astephenson Date: Sun, 29 Jun 2025 21:46:30 -0700 Subject: [PATCH 2/3] Update READM flash-attn formatting wanted to make it look nicer. --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index e7b083d4e..1c52767e1 100644 --- a/README.md +++ b/README.md @@ -66,7 +66,7 @@ uv venv openr1 --python 3.11 && source openr1/bin/activate && uv pip install --u > [!TIP] > For Hugging Face cluster users, add `export UV_LINK_MODE=copy` to your `.bashrc` to suppress cache warnings from `uv` -Next, install vLLM and FlashAttention (use version 2.7.4.post1 to avoid ABI mismatches): +Next, install vLLM and FlashAttention (use Flash Attention `v2.7.4.post1` to avoid ABI mismatches): ```shell uv pip install vllm==0.8.5.post1 From a853173b9e70f4d04e4b72856f75c9c3dd4a5f5f Mon Sep 17 00:00:00 2001 From: astephenson Date: Sun, 29 Jun 2025 21:48:17 -0700 Subject: [PATCH 3/3] Updated Makefile with Locked Flash-attn version This version utilizes flash-attn==2.7.4.post1 to prevent ABI mismatches with newer commits. --- Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile b/Makefile index 112528a19..db27bc7c4 100644 --- a/Makefile +++ b/Makefile @@ -12,7 +12,7 @@ install: . openr1/bin/activate && uv pip install --upgrade pip && \ uv pip install vllm==0.8.5.post1 && \ uv pip install setuptools && \ - uv pip install flash-attn --no-build-isolation && \ + uv pip install flash-attn==2.7.4.post1 --no-build-isolation && \ GIT_LFS_SKIP_SMUDGE=1 uv pip install -e ".[dev]" style: