A LLM semantic caching system aiming to enhance user experience by reducing response time via cached query-result pairs.
-
Updated
Jun 30, 2025 - Python
#
semantic-cache
Here are 102 public repositories matching this topic...
Language: All
Filter by language
Sort: Most stars
Sort options
Redis Vector Library (RedisVL) -- the AI-native Python client for Redis.
python search redis mcp openai embedding redis-search vector-search huggingface vector-database large-language-models llm anthropic semantic-cache retrieval-augmented-generation llmcache
-
Updated
Jun 10, 2026 - Python
mimir is a drop-in proxy that caches LLM API responses using semantic similarity, reducing costs and latency for repeated or similar queries.
-
Updated
Dec 24, 2025 - Go
SmarterRouter: An intelligent LLM gateway and VRAM-aware router for Ollama, llama.cpp, and OpenAI. Features semantic caching, model profiling, and automatic failover for local AI labs.
docker self-hosted model-serving gpu-monitoring fastapi llm openai-proxy semantic-cache local-llm ollama llm-proxy ollama-api ai-gateway llm-router self-hosted-ai ai-cache
-
Updated
May 10, 2026 - Python
Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.
mcp gateway kong guardrails pii-detection llm llmops semantic-cache prompt-management llm-proxy litellm ai-gateway ai-infrastructure llm-cost llm-strategy
-
Updated
Jun 12, 2026 - Go
Reliable and Efficient Semantic Prompt Caching with vCache
machine-learning consistency cache chatbot openai llama gpt verified memcache correctness semantic-search similarity-search rag vector-search guarantees vector-database llm vllm semantic-cache llm-memory
-
Updated
Dec 17, 2025 - Python
Redis integration for Google Agent Development Kit (ADK) - Memory, Sessions, Search Tools, MCP
memory semantic-search long-term-memory vector-search hybrid-search semantic-cache mcp-server agent-memory session-memory redis-agent-memory
-
Updated
Jun 11, 2026 - Python
Unified multi-layer caching library for AI/agent pipelines — LangChain, LangGraph, AutoGen, CrewAI, Agno, A2A
agent embeddings caching-strategies autogen rag caching-memory agno langchain semantic-cache langchain-python crewai langgraph aiagents llm-cache ai-cache gptcache-alternative
-
Updated
Jun 4, 2026 - Python
Redis Vector Library (RedisVL) -- the AI-native Java client for Redis.
java redis ai embeddings vectors rag vector-search vector-database llm generative-ai semantic-cache llm-cache rag-chatbot semantic-routing agentic-ai
-
Updated
Jun 5, 2026 - Java
Enterprise AI traffic gateway — unified compliance, routing across 20+ LLM providers, semantic cache, quotas, and audit. SDK / network / OS-layer intercept.
golang enterprise routing rate-limiting audit self-hosted bedrock gemini openai quota compliance observability pii-detection mitm-proxy llm anthropic semantic-cache ai-gateway llm-gateway openai-compatible
-
Updated
Jun 9, 2026 - Go
This is a RAG based chatbot in which semantic cache and guardrails have been incorporated.
-
Updated
Nov 11, 2024 - HTML
This repository contains sample code demonstrating how to implement a verified semantic cache using Amazon Bedrock Knowledge Bases to prevent hallucinations in Large Language Model (LLM) responses while improving latency and reducing costs.
agent aws demo bedrock rag aws-blog llm semantic-cache llm-agent amazon-bedrock amazon-bedrock-agents amazon-bedrock-knowledge-bases
-
Updated
Apr 3, 2025 - Jupyter Notebook
ToolOps is a framework-agnostic middleware SDK that treats every tool call as a first-class operation. By wrapping your tools in a single decorator, you instantly upgrade them with industrial-grade caching, resilience, and observability.
python agent redis tools ai cache prometheus python3 retry cache-storage python-sdk circuit-breaker resilience ai-agents opentelemetry langchain semantic-cache valkey agent-tools agentic-ai
-
Updated
May 30, 2026 - Python
High-performance LLM query cache with semantic search. Reduce API costs 80% and latency from 8.5s to 1ms using Redis + Qdrant vector DB. Multi-provider support (OpenAI, Anthropic).
redis embeddings openai cost-optimization rag fastapi vector-database qdrant semantic-cache llm-caching
-
Updated
Dec 2, 2025 - Python
🏆 #1 on LLM routing benchmark · Cheapest LLM router with memory · Open-source parallel multi-LLM execution across 47+ providers
nodejs benchmark typescript memory api-gateway devtools artificial-intelligence nvidia ensemble circuit-breaker cli-tool cost-optimization llm openai-proxy semantic-cache ai-gateway llm-gateway llm-routing parallel-llm routerarena
-
Updated
Jun 13, 2026 - TypeScript
Local-first semantic cache for AI agents. A small C daemon + CLI that remembers what your agent learned across sessions. Plugs into Claude Code, Codex, Gemini CLI, and Claude Desktop / ChatGPT via MCP. No LLM calls, no SaaS, no API key.
c cli caching daemon sqlite mcp embeddings knowledge-graph command-line-tool semantic-search ai-agents local-first llama-cpp semantic-cache llm-agents bge-m3 sqlite-vec agent-memory claude-code
-
Updated
May 20, 2026 - C
AI real-estate automation platform: Telegram bot, RAG, apartment search, CRM workflows, voice agent, Langfuse observability, and Dockerized AI runtime.
real-estate python docker redis telegram-bot crm ai-agents aiogram rag fastapi voice-ai hybrid-search livekit qdrant llm semantic-cache kommo langfuse langgraph bge-m3
-
Updated
Jun 12, 2026 - Python
OpenAI-compatible LLM gateway that reduces API costs using Redis exact cache and Qdrant semantic cache.
rust redis openai vector-search qdrant llm semantic-cache ai-gateway ai-infrastructure ai-cost-optimization
-
Updated
Jun 13, 2026 - Rust
Enhance LLM retrieval performance with Azure Cosmos DB Semantic Cache. Learn how to integrate and optimize caching strategies in real-world web applications.
-
Updated
Mar 22, 2024 - Python
Improve this page
Add a description, image, and links to the semantic-cache topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the semantic-cache topic, visit your repo's landing page and select "manage topics."