Skip to content

Commit 5943332

Browse files
authored
Update README.md
More explanation for our purpose for 2 docker images
1 parent bfff48f commit 5943332

File tree

1 file changed

+13
-4
lines changed

1 file changed

+13
-4
lines changed

vllm/README.md

Lines changed: 13 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -30,17 +30,26 @@ llm-scaler-vllm is an extended and optimized version of vLLM, specifically adapt
3030

3131
## 1. Getting Started and Usage
3232

33-
We provide three offerings to setup the environment and run evaluation:
33+
Overall, we provide three offerings to setup the environment and run evaluation:
3434

35-
- Bare Mental BKC Installation Script
35+
- Bare Mental BKC Installation Script (native_bkc_setup.sh)
3636
Linux kernel, GPU firmware and docker library setup
3737

38-
- Platform Evaluation Docker Image
38+
- Platform Evaluation Docker Image (llm-scaler-platform)
3939
GEMM/GPU Memory Bandwidth/P2P/1CCL benchmark
4040

41-
- vllm Inference Docker Image
41+
- vllm Inference Docker Image (llm-scaler-vllm)
4242
LLM inference evaluation
4343

44+
We provide two different Docker images, each designed for a distinct use case:
45+
46+
| Use Case | Description | Required Steps |
47+
| -------- | ----------- | -------------- |
48+
| **Platform Evaluation** | For evaluating platform capabilities only, with no intention to run vLLM inference. | 1. Install **Ubuntu 25.04** <br> 2. Run the **bare-metal BKC installation script** <br> 3. Pull the **platform evaluation Docker image** from Docker Hub (everything is pre-installed, no additional setup required) |
49+
| **vLLM Inference Benchmark** | For running inference benchmarks based on vLLM/IPEX. | 1. Install **Ubuntu 25.04** <br> 2. Run the **bare-metal BKC installation script** <br> 3. Pull the **vLLM Docker image** from Docker Hub <br> 4. Download the target model <br> 5. Run **vLLM-based inference performance tests** |
50+
51+
The platform evaluation Docker image is intended for ODM customers who primarily need a quick assessment of platform capabilities. This image only includes platform evaluation tools and is significantly smaller in size compared to the vLLM Docker image.
52+
4453
### 1.1 Install Native Environment
4554

4655
First, install a standard Ubuntu 25.04

0 commit comments

Comments
 (0)