This app leverages Semantic Caching to minimize inference latency and reduce API costs by reusing semantically similar prompt responses.
-
Updated
Jul 4, 2025 - Python
#
semantic-caching
Here are 5 public repositories matching this topic...
Official implementation of "SentenceKV: Efficient LLM Inference via Sentence-Level Semantic KV Caching" (COLM 2025). A novel KV cache compression method that organizes cache at sentence level using semantic similarity.
natural-language-processing transformers memory-efficiency efficient-inference inference-optimization kv-cache llm semantic-caching colm2025
-
Updated
Sep 29, 2025 - Python
This is a hands-on workshop to create the application Redis Movies Searcher
java docker redis full-text-search riot cache-aside vector-database vector-similarity-search semantic-caching
-
Updated
Jun 19, 2025 - Java
Improve this page
Add a description, image, and links to the semantic-caching topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the semantic-caching topic, visit your repo's landing page and select "manage topics."