feat: CUDA/TRT/CoreML execution provider + RTF benchmarks#5

Merged

wavekat-eason merged 43 commits intomainfrom

feat/cuda-provider

Apr 7, 2026

Commits on Apr 7, 2026

feat: CUDA/TensorRT/CoreML execution provider support

wavekat-eason
and
claude
committed
docs: rename 06-colab-cuda-gpu to 06-cuda-provider

wavekat-eason
and
claude
committed
docs: add Google Colab setup section to cuda-provider doc

wavekat-eason
and
claude
committed
docs: add Colab badge and execution provider flags to README

wavekat-eason
and
claude
committed
docs: remove WAV I/O note from README

wavekat-eason
and
claude
committed
docs: fix Colab glibc incompatibility with ORT_STRATEGY=system

wavekat-eason
and
claude
committed
docs: fix ORT_LIB_LOCATION to point to capi/ subdirectory

wavekat-eason
and
claude
committed
docs: create libonnxruntime.so symlink for ort-sys system strategy

wavekat-eason
and
claude
committed
docs: add shell variant for ORT setup in Colab terminal

wavekat-eason
and
claude
committed
docs: add ORT_PREFER_DYNAMIC_LINK=1 to fix static link fallback

wavekat-eason
and
claude
committed
docs: add libonnxruntime.so.1 SONAME symlink for runtime linker

wavekat-eason
and
claude
committed
docs: fix ORT API version mismatch, remove version pin

wavekat-eason
and
claude
committed
docs: use lexists to handle stale symlinks on ORT version upgrade

wavekat-eason
and
claude
committed
feat: download into WAVEKAT_MODEL_DIR when files are missing

wavekat-eason
and
claude
committed
docs: fix ORT symlink escapes model dir via cp -rL from Drive

wavekat-eason
and
claude
committed
docs: convert model copy script to Python notebook cell

wavekat-eason
and
claude
committed
feat: add --provider flag to synthesize example

wavekat-eason
and
claude
committed
fix: use SameAsRequested arena strategy for CUDA EP

wavekat-eason
and
claude
committed
fix: shim missing glibc 2.38 C23 strto* symbols for Colab

wavekat-eason
and
claude
committed
fix: error on EP failure instead of silent CPU fallback

wavekat-eason
and
claude
committed
feat: add load-dynamic feature for glibc 2.35 compat (Colab)

wavekat-eason
and
claude
committed
revert: remove glibc compat workarounds

wavekat-eason
and
claude
committed
fix: hard-link HF Hub symlinks so ORT external data validation passes

wavekat-eason
and
claude
committed
fix: hard-link symlink target, not symlink inode, for ORT compat

wavekat-eason
and
claude
committed
fix: add error_on_failure to CUDA/TRT/CoreML EPs; add Azure Ubuntu 24.04 setup docs

wavekat-eason
and
claude
committed
fix: call error_on_failure() on ExecutionProviderDispatch, not EP builder

wavekat-eason
and
claude
committed
docs: fix ORT CUDA library bundling claim; add CUDA runtime install to Azure steps

wavekat-eason
and
claude
committed
feat: replace empty-line quit with /quit command in interactive mode

wavekat-eason
and
claude
committed
feat: add bench_rtf example for qwen3-tts RTF benchmarking

wavekat-eason
and
claude
committed
feat: auto-update README benchmark table from CSV results

wavekat-eason
and
claude
committed
bench: add CPU int4 baseline results (RTF ~2.0x)

wavekat-eason
and
claude
committed
docs: add benchmarking guide (07-benchmarking.md)

wavekat-eason
and
claude
committed
docs: add TensorRT install step for Azure T4

wavekat-eason
and
claude
committed
fix: use NextPowerOfTwo arena strategy for CUDA EP

wavekat-eason
and
claude
committed
feat: self-describing CSV with backend/precision/provider/hardware/date

wavekat-eason
and
claude
committed
fix: tag cpu-int4 benchmark with Standard_NC4as_T4_v3 hardware

wavekat-eason
and
claude
committed
feat: add CUDA int4 benchmark results for Standard_NC4as_T4_v3

wavekat-eason
and
claude
committed
chore: add update-readme Make target

wavekat-eason
and
claude
committed
feat: rotate through text pool per iteration in bench_rtf

wavekat-eason
and
claude
committed
chore: add make ci target mirroring GitHub Actions

wavekat-eason
and
claude
committed
style: apply cargo fmt

wavekat-eason
and
claude
committed
fix: use is_multiple_of() in is_leap to satisfy clippy

wavekat-eason
and
claude
committed
docs: mark CUDA provider doc as implemented

wavekat-eason
and
claude
committed