Full refactor of GuideLLM #351
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… refactor branch Signed-off-by: Mark Kurtz <mark.j.kurtz@gmail.com>
markurtz
force-pushed
the
features/refactor/base
branch
from
ebdb6ce to
a2d19cd
Compare
… refactor branch Signed-off-by: Mark Kurtz <mark.j.kurtz@gmail.com>
…ng config.py to settings.py due to later config additions and potential conflicts in naming Signed-off-by: Mark Kurtz <mark.j.kurtz@gmail.com>
Signed-off-by: Mark Kurtz <mark.j.kurtz@gmail.com>
Signed-off-by: Mark Kurtz <mark.j.kurtz@gmail.com>
Signed-off-by: Mark Kurtz <mark.j.kurtz@gmail.com>
Signed-off-by: Mark Kurtz <mark.j.kurtz@gmail.com>
Signed-off-by: Mark Kurtz <mark.j.kurtz@gmail.com>
…view Signed-off-by: Mark Kurtz <mark.j.kurtz@gmail.com>
Signed-off-by: Mark Kurtz <mark.j.kurtz@gmail.com>
… for plural Signed-off-by: Mark Kurtz <mark.j.kurtz@gmail.com>
…p to avoid conflicts Signed-off-by: Mark Kurtz <mark.j.kurtz@gmail.com>
Signed-off-by: Mark Kurtz <mark.j.kurtz@gmail.com>
Signed-off-by: Mark Kurtz <mark.j.kurtz@gmail.com>
Signed-off-by: Mark Kurtz <mark.j.kurtz@gmail.com>
Signed-off-by: Mark Kurtz <mark.j.kurtz@gmail.com>
Signed-off-by: Mark Kurtz <mark.j.kurtz@gmail.com>
Signed-off-by: Mark Kurtz <mark.j.kurtz@gmail.com>
Signed-off-by: jaredoconnell <joconnel@redhat.com>
## Summary <!-- Include a short paragraph of the changes introduced in this PR. If this PR requires additional context or rationale, explain why the changes are necessary. --> TODO --- - [x] "I certify that all code in this PR is my own, except as noted below." ## Use of AI - [ ] Includes AI-assisted code completion - [ ] Includes code generated by an AI application - [ ] Includes AI-generated tests (NOTE: AI written tests should have a docstring that includes `## WRITTEN BY AI ##`)
## Summary This PR ports the new functionality from `benchmark run` to `benchmark from-file`, and does so in a way that reuses as much code as practical to have one source of truth. ## Details <!-- Provide a detailed list of all changes introduced in this pull request. --> - Fixes from-file by making it to use the new output format. - Moves code related to the new output formats to separate functions that are called from both benchmark entrypoints. - Moves additional chunks of code out of the large benchmark run entrypoint function for modularity. ## Test Plan Run a benchmark with an output of json or yaml, and use `from-file` to re-import it and export it. You can select any output type supported by `benchmark run`. `guidellm benchmark from-file ./result.json --output-formats console` `guidellm benchmark from-file ./result.yaml --output-formats yaml` ## Related Issues --- - [x] "I certify that all code in this PR is my own, except as noted below." ## Use of AI - [x] Includes AI-assisted code completion - [ ] Includes code generated by an AI application - [ ] Includes AI-generated tests (NOTE: AI written tests should have a docstring that includes `## WRITTEN BY AI ##`) --------- Signed-off-by: Jared O'Connell <joconnel@redhat.com>
## Summary Reintroduces a few changes from main --------- Signed-off-by: Samuel Monson <smonson@redhat.com>
Replace scenario entrypoint with a decorator Forward-port get_default and from_file to Scenario Apply scenario args as an update to kwargs Readd scenario support to CLI Signed-off-by: Samuel Monson <smonson@redhat.com>
Signed-off-by: Samuel Monson <smonson@redhat.com>
Signed-off-by: Samuel Monson <smonson@redhat.com>
Signed-off-by: Samuel Monson <smonson@redhat.com>
Signed-off-by: Jared O'Connell <joconnel@redhat.com>
… are not related to not found errors) for better messaging to the user
…factor/propagate_deserializer_errors
…s loading (#413) ## Summary Propagate errors not related to not found dataset errors to surface to the user that the dataset configuration is incorrect and how to address/fix it ## Details <!-- Provide a detailed list of all changes introduced in this pull request. --> - [ ] ## Test Plan <!-- List the steps needed to test this PR. --> - ## Related Issues <!-- Link any relevant issues that this PR addresses. --> - Resolves # --- - [ ] "I certify that all code in this PR is my own, except as noted below." ## Use of AI - [ ] Includes AI-assisted code completion - [ ] Includes code generated by an AI application - [ ] Includes AI-generated tests (NOTE: AI written tests should have a docstring that includes `## WRITTEN BY AI ##`)
…audio flows (#412) ## Summary Rework scheduling strategies and how workers/worker group uses them to both simplify and ensure race conditions won't happen between worker processes and grabbing timings. Additionally, fix stats accumulation bugs and data loading bugs that were causing audio to fail ## Details <!-- Provide a detailed list of all changes introduced in this pull request. --> - [ ] ## Test Plan <!-- List the steps needed to test this PR. --> - ## Related Issues <!-- Link any relevant issues that this PR addresses. --> - Resolves # --- - [ ] "I certify that all code in this PR is my own, except as noted below." ## Use of AI - [ ] Includes AI-assisted code completion - [ ] Includes code generated by an AI application - [ ] Includes AI-generated tests (NOTE: AI written tests should have a docstring that includes `## WRITTEN BY AI ##`)
… package and CLI pathways (#414) ## Summary Changed the benchmarking entrypoint to take in an Args object which is now used to load scenarios. It enables a single source of truth in addition to being able to save the exact configurations in the report output. ## Details <!-- Provide a detailed list of all changes introduced in this pull request. --> - [ ] ## Test Plan <!-- List the steps needed to test this PR. --> - ## Related Issues <!-- Link any relevant issues that this PR addresses. --> - Resolves # --- - [ ] "I certify that all code in this PR is my own, except as noted below." ## Use of AI - [ ] Includes AI-assisted code completion - [ ] Includes code generated by an AI application - [ ] Includes AI-generated tests (NOTE: AI written tests should have a docstring that includes `## WRITTEN BY AI ##`)
Signed-off-by: Samuel Monson <smonson@redhat.com>
Signed-off-by: Samuel Monson <smonson@redhat.com>
Signed-off-by: Samuel Monson <smonson@redhat.com>
Signed-off-by: Samuel Monson <smonson@redhat.com>
Signed-off-by: Samuel Monson <smonson@redhat.com>
Signed-off-by: Samuel Monson <smonson@redhat.com>
Signed-off-by: Samuel Monson <smonson@redhat.com>
Signed-off-by: Samuel Monson <smonson@redhat.com>
…odec (#411) ## TODO - [ ] ~~More flexible version locking in multimodal extras group~~ - Goal with this was to add locking for different torchcodec/torch versions but honestly its not worth the hassle - [x] Check for multi-modal libs being installed - [ ] More testing on `encode_audio` ## Summary <!-- Include a short paragraph of the changes introduced in this PR. If this PR requires additional context or rationale, explain why the changes are necessary. --> Replaces audio processing libraries with `torchcodec` which eliminates 19 dependencies and brings us inline with what HuggingFace `datasets` is doing. ## Details <!-- Provide a detailed list of all changes introduced in this pull request. --> - ## Test Plan <!-- List the steps needed to test this PR. --> - Run against audio server with ```bash guidellm benchmark run \ --target http://localhost:8000 \ --profile "synchronous" \ --max-requests 20 \ --request-type "audio_transcriptions" \ --data "openslr/librispeech_asr" \ --data-args '{"name": "clean", "split": "test"}' ``` --- - [x] "I certify that all code in this PR is my own, except as noted below." ## Use of AI - [x] Includes AI-assisted code completion - [ ] Includes code generated by an AI application - [ ] Includes AI-generated tests (NOTE: AI written tests should have a docstring that includes `## WRITTEN BY AI ##`)
Signed-off-by: Samuel Monson <smonson@redhat.com>
Signed-off-by: Samuel Monson <smonson@redhat.com>
Signed-off-by: Jared O'Connell <joconnel@redhat.com>
Signed-off-by: Jared O'Connell <joconnel@redhat.com>
Signed-off-by: Jared O'Connell <joconnel@redhat.com>
Signed-off-by: Jared O'Connell <joconnel@redhat.com>
## Summary <!-- Include a short paragraph of the changes introduced in this PR. If this PR requires additional context or rationale, explain why the changes are necessary. --> Adds a `tox` env for updating the lock file. Also allows args for mypy env. --- - [x] "I certify that all code in this PR is my own, except as noted below." ## Use of AI - [ ] Includes AI-assisted code completion - [ ] Includes code generated by an AI application - [ ] Includes AI-generated tests (NOTE: AI written tests should have a docstring that includes `## WRITTEN BY AI ##`)
## Summary Various type fixes with the goal of not breaking anything. --- - [x] "I certify that all code in this PR is my own, except as noted below." ## Use of AI - [x] Includes AI-assisted code completion - [ ] Includes code generated by an AI application - [ ] Includes AI-generated tests (NOTE: AI written tests should have a docstring that includes `## WRITTEN BY AI ##`)
sjmonson
force-pushed
the
features/refactor/base
branch
from
84bc3d8 to
fbd417f
Compare
sjmonson
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
TODO
Details
TODO
Test Plan
TODO
Related Issues
TODO