-
Notifications
You must be signed in to change notification settings - Fork 133
feat: Implement testing profile with mock vllm in docker compose #190
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: JaredforReal <w13431838023@gmail.com>
👥 vLLM Semantic Team NotificationThe following members have been identified for the changed files in this PR and have been automatically assigned: 📁
|
✅ Deploy Preview for vllm-semantic-router ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
✅ Deploy Preview for vllm-semantic-router ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: JaredforReal <w13431838023@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you, left one comment.
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Dockerfile.extproc
Outdated
COPY src/semantic-router/go.mod src/semantic-router/go.sum src/semantic-router/ | ||
COPY candle-binding/go.mod candle-binding/semantic-router.go candle-binding/ | ||
|
||
# Pre-download Go modules to leverage Docker layer caching and fail fast if mirrors are unreachable |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can you move the dockerfile optimization to another PR?
@JaredforReal lgtm, can you move the dockerfile optimization out this PR? we can review them in the followup one. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for your contribution.
There are currently 4 items related to docker. I'm wondering if we should move them to the "tools/deploy" or "tools/docker" directories? This would make the project root directory cleaner and neater.
@yuluo-yx Got u, we already have a deploy/kubernetes in the root dir, which I haven't read and tried. I will try to wrap files related to Docker (including scripts/entrypoint.sh) in a tools/docker folder and make them more graceful and easier to understand. Thanks!! |
@yuluo-yx Thanks for your advice! I have tried to move docker files to |
Signed-off-by: JaredforReal <w13431838023@gmail.com>
What type of PR is this?
feat: Implement testing profile
What this PR does / why we need it:
Implement a MVP of testing profile with mock vllm:
Docker.extproc
and 'scripts/entrypoint.sh`)tools/mock_vllm
)config/config.testing.yaml
)Which issue(s) this PR fixes:
Fixes #189
For reviewer:
This implementation, which has affected numerous files, may be worth discussing. Looking forward to any suggestions.