Solving AI Latency and Cost Problems with Valkey - Aiven Webinar
LLM providers charge you for caching, and they do it on their terms. You can run it yourself for a fraction of the cost. In this Aiven webinar, Jay Miller (Staff Developer Advocate) shows how to use semantic caching and semantic routing on Valkey to cut prompt costs + latency. You'll see a working demo, "Template Scout", that searches GitHub for deployable repos and picks the cheapest model that can answer each question. What you'll learn: Why AI costs + latency compound, and where a cache fits How a cache hit differs from a semantic hit, and how vector distance sets the threshold How semantic routing sends each prompt to a cached answer, a cheap model, or a top-tier model How Template Scout stores 500+ GitHub repos in about 30 MB on a Valkey developer tier plan (around $12/month) 🔗 Links Blog post, right-sizing model usage with Valkey + semantic routing: https://aiven.io/blog/right-sizing-your-model-usage-with-valkey-and-semantic-routing Webinar registration page: https://aiven.io/webinar/solving-real-ai-problems-using-valkey Template Scout demo repo: https://github.com/Aiven-Labs/semantic-cache-python-run-on-runtime-demo Valkey Admin on Aiven Runtime: https://github.com/aiven-labs/valkey-admin-aiven-apps Runs on Runtime templates: https://templates.aiven.io/ Valkey: https://valkey.io/ Try Aiven free for 30 days + meet Jay: https://aiven.io/jay 🎁 Claim your $10 Aiven credit (valid until November 7) In the Aiven Console (https://console.aiven.io/), go to Billing | Billing groups. Open your billing group and click Details. On the Credits tab, enter code AIVEN_VALKEY_10 and click Claim credits. Chapters 0:00 Intro 3:07 The economics of AI 6:20 Why a cache comes first (Valkey) 10:30 Semantic hits vs. semantic misses 12:50 Semantic caching 14:20 Semantic routing 16:20 Tools: BetterDB, Langfuse, LiteLLM, OpenRouter, Semantic Router 18:20 Demo: Template Scout on Aiven Runtime 23:40 Demo: chat + model routing 27:00 Inside Valkey: cached repos + conversations 31:50 Q&A: embedding models CONNECT WITH US Website: http://aiven.io LinkedIn: https://www.linkedin.com/company/aiven/ GitHub: https://github.com/aiven X: https://twitter.com/aiven_io