TurboQuant KV cache compression for MLX with fused Metal kernels. 4.6x compression at 98% FP16 speed.
-
Updated
Mar 29, 2026 - Python
#
turboquant
Here are 33 public repositories matching this topic...
First open-source TurboQuant KV cache compression for LLM inference. Drop-in for HuggingFace. pip install turboquant.
machine-learning compression gpu transformers inference pytorch quantization vram huggingface kv-cache llm turboquant
-
Updated
Mar 27, 2026 - Python
Near-optimal vector quantization from Google's ICLR 2026 paper — 95% recall, 5x compression, zero preprocessing, pure Python FAISS replacement
python machine-learning information-retrieval compression deep-learning numpy quantization semantic-search similarity-search vector-quantization faiss rag vector-database kv-cache ann-search llm approximate-nearest-neighbor embedding-compression iclr-2026 turboquant
-
Updated
Mar 28, 2026 - Python
Minimal, zero-dependency LLM inference in pure C11. CPU-first with NEON/AVX2 SIMD. Flash MoE (pread + LRU expert cache). TurboQuant 3-bit KV compression (8.9x less memory per session). 20+ GGUF quant formats. Compiles to WASM.
-
Updated
Mar 28, 2026 - C
No bs theatricals. Real automated pentesting. Mac only.
api caching engine hacking openai pentesting vulnerability-detection vulnerability-scanners mlx uncensored hacking-tools pentesting-tools llm llms anthropic automated-pentesting mlxllm mlxstudio vmlx turboquant
-
Updated
Mar 26, 2026 - Python
TurboQuant: Native 3-Bit Quantization for Ollama - Achieve 25-28% better compression than Q4_0 while maintaining high-speed CPU inference. Experimentally integrated into Ollama with custom GGML kernels for LLM efficiency.
-
Updated
Mar 28, 2026 - Go
Hardware-agnostic machine learning infrastructure for .NET. Implements high-performance neural network layers in C# that are transpiled to run on WebGPU, CUDA, OpenCL, WebGL, CPU, and Wasm via SpawnDev.ILGPU. Optimized for Blazor WebAssembly and native GPU execution.
webgl machine-learning opencl distributed-computing cuda webassembly pytorch webgl2 tensroflow webgpu inference-engine model-training coreml onnx blazor tflite blazor-webassembly safetensors gguf turboquant
-
Updated
Mar 29, 2026 - WGSL
TurboQuant‑style embedding compression for RAG: an SDK using fixed rotations, PolarQuant, and QJL residual sketches for compact storage and fast similarity search
-
Updated
Mar 28, 2026 - Python
KV Cache with PagedAttention vs PagedAttention + TurboQuant - experiments across token sizes comparing memory, latency, and accuracy.
inference large-language-models llm llms llm-inference kvcache kvcache-optimization kvcache-compression turboquant
-
Updated
Mar 26, 2026 - Python
TurboQuant (ICLR 2026) ported to Apple Silicon — KV cache compression with MLX Metal kernels + PyTorch CPU
machine-learning deep-learning metal transformers inference pytorch attention quantization mlx iclr kv-cache apple-silicon llm llm-inference turboquant
-
Updated
Mar 28, 2026 - Python
Near-optimal vector quantization for LLM KV cache compression. Python implementation of TurboQuant (ICLR 2026) — PolarQuant + QJL for 3-bit quantization with minimal accuracy loss and up to 8x memory reduction.
python deep-learning numpy transformer attention quantization vector-quantization model-compression inference-optimization memory-optimization kv-cache post-training-quantization llm llm-inference llm-compression kv-cache-compression iclr-2026 turboquant polarquant qjl
-
Updated
Mar 28, 2026 - Python
AI agent skill implementing Google's TurboQuant compression algorithm (ICLR 2026) — 6x KV cache memory reduction, 8x speedup, zero accuracy loss. Compatible with Claude Code, Codex CLI, and all Agent Skills-compatible tools.
python deep-learning transformer quantization codex vector-quantization model-compression inference-optimization kv-cache ai-agent llm llm-compression kv-cache-compression claude-code coding-agent iclr-2026 agent-skill turboquant polarquant qjl
-
Updated
Mar 28, 2026 - Python
Interactive Benchmarking Tool for TurboQuant KV Cache Compression. Supports 2-4 bit quantization with Real-time Metrics
nlp machine-learning deep-learning pytorch transformer mistral vector-quantization model-compression inference-optimization kv-cache llm vllm qwen iclr-2026 turboquant
-
Updated
Mar 28, 2026 - Python
AI Code Review Memory - learns from your team's bug history and warns when similar patterns appear
-
Updated
Mar 28, 2026 - Python
Implementation of https://research.google/blog/turboquant-redefining-ai-efficiency-with-extreme-compression/
-
Updated
Mar 30, 2026 - C++
Turbo Index
google hpc gpu information-theory pytorch nearest-neighbor-search quantization vector-quantization kv-cache large-language-models llm-inference turboquant turboindexer
-
Updated
Mar 25, 2026 - Python
Experimental TurboQuant implementation and llama.cpp-style integration path for long-context inference
-
Updated
Mar 29, 2026 - C++
ChatMind: Semantic search for Discord & KakaoTalk chat messages. Search by meaning, not keywords. Powered by TurboQuant compression (ICLR 2026).
multilingual python nlp discord embeddings developer-tools semantic-search kakaotalk cli-tool sentence-embeddings chat-export iclr2026 natural-language-search turboquant vector-compression chat-search message-search
-
Updated
Mar 28, 2026 - Python
CommitMind: Semantic search for Git commit history powered by TurboQuant vector compression (ICLR 2026). Search commits by meaning, not just keywords.
python git nlp machine-learning embeddings code-search developer-tools quantization semantic-search git-history cli-tool sentence-embeddings commit-history commit-search iclr2026 natural-language-search turboquant vector-compression
-
Updated
Mar 28, 2026 - Python
Improve this page
Add a description, image, and links to the turboquant topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the turboquant topic, visit your repo's landing page and select "manage topics."