Skip to content

refactor cuda graph runner #11

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 67 commits into
base: main
Choose a base branch
from
Open

refactor cuda graph runner #11

wants to merge 67 commits into from

Conversation

QiJune
Copy link
Owner

@QiJune QiJune commented Aug 11, 2025

No description provided.

QiJune added 30 commits July 30, 2025 17:32
@QiJune
add CUDAGraphModelEngine
31a15db 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
rebase
52fd48a 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
rebase
6d9673d 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
rebase
fe6fa01 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
rebase
3841576 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
update
57b9db1 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
fix
fdfacbc 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
fix
2eb3030 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
fix
0640458 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
fix
4fb2876 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
fix
a0d4d52 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
fix
96793c8 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
fix
df4d782 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
fix
677c218 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
fix
5b1ed33 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
fix
215d0d5 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
update
cf36b99 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
rebase
66a0398 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
rename
492f6c0 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
rebase
91e87a5 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
fix
7b470e9 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
clean
d821fa0 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
polish
22df791 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
fix
c10dba4 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
fix
ed19860 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
fix
1f64f35 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
fix
c13d9fb 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
fix
e20fbab 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
fix
fd5d5cb 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
rebase
73a8c90 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
QiJune and others added 30 commits August 15, 2025 08:55
@QiJune
fix
801c3d4 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
clean moe
6690c36 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
clean
48eab08 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
fix
f8c12ec 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
fix
16ebd43 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
fix test_pad_generation_requests
9d8f9f6 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
Merge branch 'main' into cuda_graph
70b4d50
@QiJune
rebase
0a08c5a 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
Merge branch 'main' into cuda_graph
50c7c22
@QiJune
rebase
3ff27b3 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
fix attention dp
b97069a 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
polish
95f488b 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
fix
b57ba3b 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
fix
9fae269 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
clean
debca3c 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
Merge branch 'main' into cuda_graph
544f41a
@QiJune
fix conflicts
e69cf93 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
rebase
afe209f 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
Merge branch 'main' into cuda_graph
3307c43
@QiJune
clean tests
43b18bd 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@QiJune
rebase
530ab91 
Signed-off-by: junq <22017000+QiJune@users.noreply.github.com>
@yiqingy0
[None][chore] Bump version to 1.1.0rc2 (NVIDIA#7167)
907bc22 
Signed-off-by: Yiqing Yan <yiqingy@nvidia.com>
@suyoggupta
[NVIDIA#7136][feat] trtllm-serve + autodeploy integration (NVIDIA#7141)
e3de575 
Signed-off-by: Suyog Gupta <41447211+suyoggupta@users.noreply.github.com>
@tomeras91 @Copilot @coderabbitai
[TRTLLM-4921][feat] Enable chunked prefill for Nemotron-H (NVIDIA#6334)
c232ba8 
Signed-off-by: Tomer Asida <57313761+tomeras91@users.noreply.github.com>
Signed-off-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
@Funatiq
[None][refactor] Simplify decoder state initialization for speculativ…
37543a9 
…e decoding (NVIDIA#6869)

Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>
@IzzyPutterman @mikeiovine
[None][feat] Deepseek: Start Eagle work (NVIDIA#6210)
b36460d 
Signed-off-by: Izzy Putterman <iputterman@nvidia.com>
Co-authored-by: Mike Iovine <miovine@nvidia.com>
@FrankD412
[None][fix] Correct KV cache percentage report out. (NVIDIA#7102)
81fd468 
Signed-off-by: Frank Di Natale <3429989+FrankD412@users.noreply.github.com>
@gracehonv
[None] [feat] nsys profile output kernel classifier (NVIDIA#7020)
3d54a1a 
Signed-off-by: Grace Ho <grho@nvidia.com>
@Tabrizian
[None][fix] Waive test (NVIDIA#7185)
96ff82e 
Signed-off-by: Iman Tabrizian <10105175+Tabrizian@users.noreply.github.com>
@QiJune
Merge branch 'main' into cuda_graph
7ff0405
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
9 participants
@QiJune @yiqingy0 @suyoggupta @tomeras91 @Funatiq @IzzyPutterman @FrankD412 @gracehonv @Tabrizian