Skip to content

Commit 6d6f2e6

Browse files
fcostaoliveirafcostaoliveira
committed
feat: comprehensive Docker improvements and precision summary enhancements
- Simplify Docker tagging strategy to only latest and version tags - Change Docker image repository from filipe958/vector-db-benchmark to redis/vector-db-benchmark - Fix Docker volume permission issues with smart entrypoint script - Add precision_summary field with clean QPS, P50, P95 metrics - Update Python version support to include 3.13 - Add comprehensive Redis environment variables (REDIS_AUTH, REDIS_USER, REDIS_CLUSTER) - Create Docker-specific README with Redis 8.2 examples - Update all documentation to use redis/vector-db-benchmark consistently - Enhance GitHub Actions with Redis CLI installation and Poetry support
1 parent 6ebf338 commit 6d6f2e6

File tree

7 files changed

+172
-50
lines changed

7 files changed

+172
-50
lines changed

‎.github/workflows/docker-publish-master.yml‎

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
name: Docker Publish - update.redisearch branch
1+
name: Docker Publish - Latest
22

33
on:
44
push:
@@ -11,7 +11,7 @@ on:
1111

1212
env:
1313
REGISTRY: docker.io
14-
IMAGE_NAME: filipe958/vector-db-benchmark
14+
IMAGE_NAME: redis/vector-db-benchmark
1515

1616
jobs:
1717
docker-publish:
@@ -61,8 +61,6 @@ jobs:
6161
images: ${{ env.IMAGE_NAME }}
6262
tags: |
6363
type=raw,value=latest
64-
type=raw,value=update-redisearch-{{sha}}
65-
type=raw,value=update-redisearch-{{date 'YYYYMMDD-HHmmss'}}
6664
6765
- name: Build and push Docker image
6866
uses: docker/build-push-action@v5
@@ -95,4 +93,4 @@ jobs:
9593
echo "docker run --rm ${{ env.IMAGE_NAME }}:latest run.py --help" >> $GITHUB_STEP_SUMMARY
9694
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY
9795
echo "" >> $GITHUB_STEP_SUMMARY
98-
echo "🔗 [View on Docker Hub](https://hub.docker.com/r/filipe958/vector-db-benchmark)" >> $GITHUB_STEP_SUMMARY
96+
echo "🔗 [View on Docker Hub](https://hub.docker.com/r/redis/vector-db-benchmark)" >> $GITHUB_STEP_SUMMARY

‎.github/workflows/docker-publish-release.yml‎

Lines changed: 2 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ on:
66

77
env:
88
REGISTRY: docker.io
9-
IMAGE_NAME: filipe958/vector-db-benchmark
9+
IMAGE_NAME: redis/vector-db-benchmark
1010

1111
jobs:
1212
docker-publish:
@@ -57,9 +57,6 @@ jobs:
5757
tags: |
5858
type=ref,event=tag
5959
type=semver,pattern={{version}}
60-
type=semver,pattern={{major}}.{{minor}}
61-
type=semver,pattern={{major}}
62-
type=raw,value=latest,enable={{is_default_branch}}
6360
6461
- name: Build and push Docker image
6562
uses: docker/build-push-action@v5
@@ -110,5 +107,5 @@ jobs:
110107
echo "docker run --rm ${{ env.IMAGE_NAME }}:latest run.py --help" >> $GITHUB_STEP_SUMMARY
111108
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY
112109
echo "" >> $GITHUB_STEP_SUMMARY
113-
echo "🔗 [View on Docker Hub](https://hub.docker.com/r/filipe958/vector-db-benchmark)" >> $GITHUB_STEP_SUMMARY
110+
echo "🔗 [View on Docker Hub](https://hub.docker.com/r/redis/vector-db-benchmark)" >> $GITHUB_STEP_SUMMARY
114111
echo "🔒 [Security Scan Results](https://github.com/${{ github.repository }}/security/code-scanning)" >> $GITHUB_STEP_SUMMARY

‎DOCKER_README.md‎

Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
# Redis Vector Database Benchmark
2+
3+
A comprehensive benchmarking tool for vector databases, including Redis (both RediSearch and Vector Sets), Weaviate, Milvus, Qdrant, OpenSearch, Postgres, and others...
4+
5+
## Quick Start
6+
7+
```bash
8+
# Pull the latest image
9+
docker pull redis/vector-db-benchmark:latest
10+
11+
# Run with help
12+
docker run --rm redis/vector-db-benchmark:latest run.py --help
13+
14+
# Check available datasets
15+
docker run --rm redis/vector-db-benchmark:latest run.py --describe datasets
16+
17+
# Basic Redis benchmark (requires local Redis)
18+
docker run --rm -v $(pwd)/results:/app/results --network=host \
19+
redis/vector-db-benchmark:latest \
20+
run.py --host localhost --engines redis-default-simple --dataset random-100
21+
```
22+
23+
## Features
24+
25+
- **42+ Datasets**: Pre-configured datasets from 25 to 1B+ vectors
26+
- **Multiple Engines**: Redis, Qdrant, Weaviate, Milvus, and more
27+
- **Real-time Monitoring**: Live performance metrics during benchmarks
28+
- **Precision Analysis**: Detailed accuracy vs performance trade-offs
29+
- **Easy Discovery**: `--describe` commands for datasets and engines
30+
31+
## Available Tags
32+
33+
- `latest` - Latest development build from update.redisearch branch
34+
35+
## Redis quick start
36+
37+
### Redis 8.2 with RediSearch
38+
```bash
39+
# Start Redis 8.2 with built-in vector support
40+
docker run -d --name redis-test -p 6379:6379 redis:8.2-rc1-bookworm
41+
42+
# Run benchmark
43+
docker run --rm -v $(pwd)/results:/app/results --network=host \
44+
redis/vector-db-benchmark:latest \
45+
run.py --host localhost --engines redis-default-simple --dataset glove-25-angular
46+
```
47+
48+
49+
## Common Usage Patterns
50+
51+
### Explore Available Options
52+
```bash
53+
# List all datasets
54+
docker run --rm redis/vector-db-benchmark:latest run.py --describe datasets
55+
56+
# List all engines
57+
docker run --rm redis/vector-db-benchmark:latest run.py --describe engines
58+
```
59+
60+
### Run Benchmarks
61+
```bash
62+
# Quick test with small dataset
63+
docker run --rm -v $(pwd)/results:/app/results --network=host \
64+
redis/vector-db-benchmark:latest \
65+
run.py --host localhost --engines redis-default-simple --dataset random-100
66+
67+
# Comprehensive benchmark with multiple configurations
68+
docker run --rm -v $(pwd)/results:/app/results --network=host \
69+
redis/vector-db-benchmark:latest \
70+
run.py --host localhost --engines "*redis*" --dataset glove-25-angular
71+
72+
# With Redis authentication
73+
docker run --rm -v $(pwd)/results:/app/results --network=host \
74+
-e REDIS_AUTH=mypassword -e REDIS_USER=myuser \
75+
redis/vector-db-benchmark:latest \
76+
run.py --host localhost --engines redis-default-simple --dataset random-100
77+
```
78+
79+
### Results Analysis
80+
```bash
81+
# View precision summary
82+
jq '.precision_summary' results/*-summary.json
83+
84+
# View detailed results
85+
jq '.search' results/*-summary.json
86+
```
87+
88+
## Volume Mounts
89+
90+
- `/app/results` - Benchmark results (JSON files)
91+
- `/app/datasets` - Dataset storage (optional, auto-downloaded)
92+
93+
## Environment Variables
94+
95+
- `REDIS_HOST` - Redis server hostname (default: localhost)
96+
- `REDIS_PORT` - Redis server port (default: 6379)
97+
- `REDIS_AUTH` - Redis password (default: None)
98+
- `REDIS_USER` - Redis username (default: None)
99+
- `REDIS_CLUSTER` - Enable Redis cluster mode (default: 0)
100+
101+
## Performance Tips
102+
103+
1. **Use `--network=host`** for best performance with local Redis
104+
2. **Mount results volume** to persist benchmark data
105+
3. **Start with small datasets** (random-100, glove-25-angular) for testing
106+
4. **Use wildcard patterns** to test multiple configurations: `--engines "*-m-16-*"`
107+
108+
## Example Output
109+
110+
```json
111+
{
112+
"precision_summary": {
113+
"0.91": {
114+
"qps": 1924.5,
115+
"p50": 49.828,
116+
"p95": 58.427
117+
},
118+
"0.94": {
119+
"qps": 1819.9,
120+
"p50": 51.68,
121+
"p95": 66.83
122+
}
123+
}
124+
}
125+
```
126+
127+
## Support
128+
129+
- **GitHub**: [redis-performance/vector-db-benchmark](https://github.com/redis-performance/vector-db-benchmark)
130+
- **Issues**: Report bugs and feature requests on GitHub
131+
- **Documentation**: Full documentation available in the repository
132+
133+
## License
134+
135+
This project is licensed under the MIT License - see the repository for details.

‎DOCKER_SETUP.md‎

Lines changed: 12 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Docker Setup and Publishing Guide
22

3-
This guide explains how to set up Docker publishing for the `vector-db-benchmark` project to Docker Hub repository `filipe958/vector-db-benchmark`.
3+
This guide explains how to set up Docker publishing for the `vector-db-benchmark` project to Docker Hub repository `redis/vector-db-benchmark`.
44

55
## 🔐 Required GitHub Secrets
66

@@ -57,11 +57,8 @@ Once secrets are configured, Docker images will be automatically published:
5757

5858
### Example Tags for Release v1.2.3
5959
```
60-
filipe958/vector-db-benchmark:v1.2.3
61-
filipe958/vector-db-benchmark:1.2.3
62-
filipe958/vector-db-benchmark:1.2
63-
filipe958/vector-db-benchmark:1
64-
filipe958/vector-db-benchmark:latest
60+
redis/vector-db-benchmark:v1.2.3
61+
redis/vector-db-benchmark:latest
6562
```
6663

6764
## 🛠️ Manual Building and Publishing
@@ -100,32 +97,32 @@ export DOCKER_PASSWORD=your_access_token
10097
### Pull and Run
10198
```bash
10299
# Latest version
103-
docker pull filipe958/vector-db-benchmark:latest
104-
docker run --rm filipe958/vector-db-benchmark:latest run.py --help
100+
docker pull redis/vector-db-benchmark:latest
101+
docker run --rm redis/vector-db-benchmark:latest run.py --help
105102

106103
# Specific version
107-
docker pull filipe958/vector-db-benchmark:v1.2.3
108-
docker run --rm filipe958/vector-db-benchmark:v1.2.3 run.py --help
104+
docker pull redis/vector-db-benchmark:v1.2.3
105+
docker run --rm redis/vector-db-benchmark:v1.2.3 run.py --help
109106
```
110107

111108
### Example Usage
112109
```bash
113110
# Basic Redis benchmark
114-
docker run --rm --network=host filipe958/vector-db-benchmark:latest \
111+
docker run --rm --network=host redis/vector-db-benchmark:latest \
115112
run.py --host localhost --engines redis --dataset random-100 --experiment redis-default-simple
116113

117114
# With custom Redis host
118-
docker run --rm filipe958/vector-db-benchmark:latest \
115+
docker run --rm redis/vector-db-benchmark:latest \
119116
run.py --host redis-server --engines redis --dataset random-100 --experiment redis-default-simple
120117

121118
# With results output (mount current directory)
122119
docker run --rm -v $(pwd)/results:/app/results --network=host \
123-
filipe958/vector-db-benchmark:latest \
120+
redis/vector-db-benchmark:latest \
124121
run.py --host localhost --engines redis --dataset random-100 --experiment redis-default-simple
125122

126123
# Using with Redis container
127124
docker run -d --name redis-test -p 6379:6379 redis:8.2-rc1-bookworm
128-
docker run --rm --network=host filipe958/vector-db-benchmark:latest \
125+
docker run --rm --network=host redis/vector-db-benchmark:latest \
129126
run.py --host localhost --engines redis --experiment redis-default-simple
130127
docker stop redis-test && docker rm redis-test
131128
```
@@ -145,7 +142,7 @@ docker stop redis-test && docker rm redis-test
145142
- Prevents merging PRs with broken Docker builds
146143

147144
### Docker Hub
148-
- View images at: https://hub.docker.com/r/filipe958/vector-db-benchmark
145+
- View images at: https://hub.docker.com/r/redis/vector-db-benchmark
149146
- Check image sizes and platforms
150147
- Review vulnerability scan results
151148

‎README.md‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -25,17 +25,17 @@ The easiest way to run vector-db-benchmark is using Docker. We provide pre-built
2525

2626
```bash
2727
# Pull the latest image
28-
docker pull filipe958/vector-db-benchmark:latest
28+
docker pull redis/vector-db-benchmark:latest
2929

3030
# Run with help
31-
docker run --rm filipe958/vector-db-benchmark:latest run.py --help
31+
docker run --rm redis/vector-db-benchmark:latest run.py --help
3232

3333
# Check which datasets are available
34-
docker run --rm filipe958/vector-db-benchmark:latest run.py --describe datasets
34+
docker run --rm redis/vector-db-benchmark:latest run.py --describe datasets
3535

3636
# Basic Redis benchmark with local Redis
3737
docker run --rm -v $(pwd)/results:/app/results --network=host \
38-
filipe958/vector-db-benchmark:latest \
38+
redis/vector-db-benchmark:latest \
3939
run.py --host localhost --engines redis-default-simple --datasets glove-25-angular
4040

4141
# At the end of the run, you will find the results in the `results` directory. Lets open the summary one, in the precision summary
@@ -76,7 +76,7 @@ docker run -d --name redis-test -p 6379:6379 redis:8.2-rc1-bookworm
7676
# Run benchmark against Redis
7777

7878
docker run --rm -v $(pwd)/results:/app/results --network=host \
79-
filipe958/vector-db-benchmark:latest \
79+
redis/vector-db-benchmark:latest \
8080
run.py --host localhost --engines redis-default-simple --dataset random-100
8181

8282
# Or use the convenience script
@@ -89,7 +89,7 @@ docker stop redis-test && docker rm redis-test
8989

9090
### Available Docker Images
9191

92-
- **Latest**: `filipe958/vector-db-benchmark:latest`
92+
- **Latest**: `redis/vector-db-benchmark:latest`
9393

9494
For detailed Docker setup and publishing information, see [DOCKER_SETUP.md](DOCKER_SETUP.md).
9595

@@ -186,7 +186,7 @@ python run.py --engines "*-m-16-*" --dataset "glove-*"
186186

187187
# Docker usage (recommended)
188188
docker run --rm -v $(pwd)/results:/app/results --network=host \
189-
filipe958/vector-db-benchmark:latest \
189+
redis/vector-db-benchmark:latest \
190190
run.py --host localhost --engines redis-default-simple --dataset random-100
191191

192192
# Get help

‎docker-build.sh‎

Lines changed: 10 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66
set -e
77

88
# Default values
9-
IMAGE_NAME="filipe958/vector-db-benchmark"
9+
IMAGE_NAME="redis/vector-db-benchmark"
1010
TAG="latest"
1111
PLATFORM=""
1212
PUSH=false
@@ -35,18 +35,18 @@ usage() {
3535
echo "Usage: $0 [OPTIONS]"
3636
echo ""
3737
echo "Options:"
38-
echo " -n, --name NAME Docker image name (default: redis-performance/vector-db-benchmark)"
38+
echo " -n, --name NAME Docker image name (default: redis/vector-db-benchmark)"
3939
echo " -t, --tag TAG Docker image tag (default: latest)"
4040
echo " -p, --platform PLATFORM Target platform (e.g., linux/amd64,linux/arm64)"
4141
echo " --push Push image to Docker Hub after building"
4242
echo " -h, --help Show this help message"
4343
echo ""
4444
echo "Examples:"
45-
echo " $0 # Build with defaults"
46-
echo " $0 -t v1.0.0 --push # Build and push with custom tag"
45+
echo " $0 # Build with defaults (latest tag)"
46+
echo " $0 -t v1.0.0 --push # Build and push version tag"
4747
echo " $0 -p linux/amd64,linux/arm64 --push # Multi-platform build and push"
4848
echo ""
49-
echo "Docker Hub Repository: redis-performance/vector-db-benchmark"
49+
echo "Docker Hub Repository: redis/vector-db-benchmark"
5050
}
5151

5252
# Parse command line arguments
@@ -80,13 +80,8 @@ while [[ $# -gt 0 ]]; do
8080
esac
8181
done
8282

83-
# Get Git information
84-
print_info "Gathering Git information..."
85-
GIT_SHA=$(git rev-parse HEAD 2>/dev/null || echo "unknown")
86-
GIT_DIRTY=$(git diff --no-ext-diff 2>/dev/null | wc -l || echo "0")
87-
88-
print_info "Git SHA: $GIT_SHA"
89-
print_info "Git Dirty: $GIT_DIRTY"
83+
# Prepare for build
84+
print_info "Preparing Docker build..."
9085

9186
# Build Docker image
9287
FULL_IMAGE_NAME="${IMAGE_NAME}:${TAG}"
@@ -119,8 +114,8 @@ else
119114
BUILD_CMD="docker build"
120115
fi
121116

122-
# Add build arguments and tags
123-
BUILD_CMD="$BUILD_CMD --build-arg GIT_SHA=$GIT_SHA --build-arg GIT_DIRTY=$GIT_DIRTY -t $FULL_IMAGE_NAME ."
117+
# Add tags
118+
BUILD_CMD="$BUILD_CMD -t $FULL_IMAGE_NAME ."
124119

125120
print_info "Executing: $BUILD_CMD"
126121

@@ -173,7 +168,7 @@ if eval $BUILD_CMD; then
173168
echo " docker run --rm --network=host $FULL_IMAGE_NAME run.py --host localhost --engines redis"
174169
echo ""
175170
if [[ "$PUSH" == "true" ]]; then
176-
print_info "Image available on Docker Hub: https://hub.docker.com/r/redis-performance/vector-db-benchmark"
171+
print_info "Image available on Docker Hub: https://hub.docker.com/r/redis/vector-db-benchmark"
177172
fi
178173
else
179174
print_error "❌ Docker build failed"

0 commit comments

Comments
 (0)