v0.2.10: Dynamic cuBLASLt loading + CUDA Graph optimizations#92

Merged

m96-chan merged 49 commits intomainfrom

feature/v0.2.10

Dec 18, 2025

Commits on Dec 16, 2025

docs(llm): update for v0.2.9 unified interface

m96-chan
and
claude
committed
feat(cutlass): add SM100/SM120 Blackwell kernel infrastructure

m96-chan
and
claude
committed
fix(cutlass): disable SM100/SM120 kernels - CUTLASS 4.3.3 only supports FP8

m96-chan
and
claude
committed
ci(windows): explicitly use CUDA 13.1 for CUTLASS 4.x compatibility

m96-chan
and
claude
committed
feat(cutlass): re-enable SM100 kernels, keep SM120 disabled

m96-chan
and
claude
committed
feat(rope): add native FP16/BF16 RoPE kernel support (#84)

m96-chan
and
claude
committed
feat(kv-cache): GPU KV Cache to eliminate CPU-GPU transfers (#83)

m96-chan
and
claude
committed
feat(attention): GPU Attention for Decode - unify all paths to GPU (#81)

m96-chan
and
claude
committed
bench: add profile_blocks.py for GPU memory analysis

m96-chan
and
claude
committed
perf(llm): add weight repacking to fix GPU memory placement (2.6x speedup)

m96-chan
and
claude
committed
feat(llm): add streaming generation (#89)

m96-chan
and
claude
committed
feat(llm): add chat template support (#90)

m96-chan
and
claude
committed
feat(attention): add Flash Attention 2 (#82)

m96-chan
and
claude
committed
feat(quantize): add INT8 weight quantization (#85)

m96-chan
and
claude
committed
feat(attention): add Paged Attention for efficient KV cache (#87)

m96-chan
and
claude
committed
feat(batch): add Continuous Batching infrastructure (#86)

m96-chan
and
claude
committed
docs(examples): add v0.2.10 comprehensive feature demo

m96-chan
and
claude
committed
feat(cuda-graph): add CUDA Graph capture/replay infrastructure

m96-chan
and
claude
committed
feat(cuda-graph): add fixed-length KV cache and SDPA context_len support

m96-chan
and
claude
committed
feat(llm): add generate_cuda_graph with fixed-length KV cache

m96-chan
and
claude
committed
perf(llm): optimize GQA with pre-expanded KV cache

m96-chan
and
claude
committed
refactor(llm): simplify generate_cuda_graph, document CUDA Graph limitations

m96-chan
and
claude
committed
feat(cuda-graph): add zero-alloc decode infrastructure

m96-chan
and
claude
committed
feat(cuda-graph): add out parameter to transpose_3d_021 and reshape_copy

m96-chan
and
claude
committed

Commits on Dec 17, 2025

Commits on Dec 18, 2025