I Built an Auto Router for Local LLMs with LiteLLM + Pi Coding Agent
In this video, I build an Auto Router for multiple local LLMs using LiteLLM, Pi Coding Agent, Ollama, and oMLX. Instead of sending every request to the biggest model, the router classifies the task and decides whether it should go to a smaller, faster model or a larger, more capable one. The setup uses models like Qwen3.5 4B, Gemma 4, and Qwen3.8 27B, running across Ollama and oMLX. I also show the difference between LiteLLM’s heuristic classifier and an LLM-powered classifier, and how this kind of routing can reduce unnecessary memory pressure and improve overall local AI throughput on Apple Silicon. Chapters: 00:00 - Introduction 00:40 - Demo of How Smart Router Works 02:35 - Why LiteLLM Router is Needed? 07:17 - Installing LiteLLMs with Docker Setup 08:59 - LiteLLM Config along with Docker Compose 13:07 - LiteLLM UI Dashboard 16:35 - Using Pi Coding Agent AutoRouter of LiteLLM My AI related Courses for you to empower today: Learn my latest course launched this week on ZERO Code AI Tests: Automate Software Tests in Plain English: https://www.udemy.com/course/zero-code-ai-testing/?couponCode=LEARN_AUG_26 Use code: LEARN_AUG_26 for 85% OFF (limited time) My Other AI related Courses for you to empower today: To Learn more about building your own AI Agents and Testing AI Agents, follow the courses below available in Udemy with Discount coupon codes. Use coupon code : LEARN_AUG_26 🤖 AI-Driven Test Automation: Playwright, Selenium, LLMs & More https://www.udemy.com/course/ai-test-automation-selenium-playwright-llms/?couponCode=LEARN_AUG_26 🤖 https://www.udemy.com/course/genai-multi-agent-software-testing/?couponCode=LEARN_AUG_26 🤖Test AI & LLM App with DeepEval, RAGAs & more using Ollama https://www.udemy.com/course/ai-testing-deepeval-ragas-ollama/?couponCode=LEARN_AUG_26 🤖Build & Test AI Agents, ChatBot, RAG with Ollama & Local LLM https://www.udemy.com/course/build-ai-agent-chatbot-rag-langchain-local-llm/?couponCode=LEARN_AUG_26 🤖2025 - Understand ,Test ,Fine-tune AI Model with HuggingFace https://www.udemy.com/course/ai-with-huggingface/?couponCode=LEARN_AUG_26 🤖2025 - Using Generative AI in Software Automation Testing https://www.udemy.com/course/generative-ai-in-software-automation-testing/?couponCode=LEARN_AUG_26 🤖AgentQL: https://youtu.be/XzmkFSqTgtM 🌐 Local Deepseek R1 for BrowserUse - https://youtu.be/NvClPPI4sK0 🌐 Browser Use - https://youtu.be/O1GzsqJFpuI ➜ Discount coupon codes for all the courses: LEARN_AUG_26 ► [Advanced Framework development Course in Selenium C#] https://www.udemy.com/course/framework-development-with-selenium-csharp-advanced/ ► [Advanced Framework development Course in Playwright C#] https://www.udemy.com/course/framework-development-with-playwright-dotnet/ #macbookpro #Qwen38 #FlashNext #LocalAI #LLM #MoE #AppleSilicon #llamaCPP #Unsloth #ClaudeOpus #ExecuteAutomation ► [Udemy] https://www.udemy.com/user/karthik-kk/ ► [XUnit with Selenium] https://www.youtube.com/playlist?list=PL6tu16kXT9Prbp7hTEb-M-luddsRJZKJ2 ► [Git Basics] https://www.youtube.com/playlist?list=PL6tu16kXT9PqIcfMOE9hX81cVOnmiM1E2 ► [SpringBoot for Testing] https://www.youtube.com/playlist?list=PL6tu16kXT9PrDr6kMGQ-CgnvCsFxrq1eS Selenium and C# ► [C# for automation testing] https://www.youtube.com/playlist?list=PL6tu16kXT9Pp3NFZgLbPZXEykeGQwxGSx ► [Selenium with C#] https://www.youtube.com/watch?v=mluLgBywW0Y&list=PL6tu16kXT9PqKSouJUV6sRVgmcKs-VCqo ► [BDD with Specflow] https://www.youtube.com/playlist?list=PL6tu16kXT9Pp3wrsaYyNRnK1QkvVv6qdI ► [BDD with Selenium] https://www.youtube.com/playlist?list=PL6tu16kXT9PrAjLHMTtttMybnV1kDpCXk ► [Selenium .NET Core] https://www.youtube.com/playlist?list=PL6tu16kXT9PqLJfuuLcvHnE61pzD8Dq9M Selenium &Java ► [Cucumber with Selenium] https://www.youtube.com/playlist?list=PL6tu16kXT9PpteusHGISu_lHcV6MbBtA6 ► [Cucumber with Selenium] https://www.youtube.com/playlist?list=PL6tu16kXT9Pqr70SZlwcmTSAfOw_0Qj3R ► [Cucumber 4 Upgrade] https://www.youtube.com/playlist?list=PL6tu16kXT9PoT4gRr5F00r1lPaTszk_-q ► [Selenium Grid] https://www.youtube.com/playlist?list=PL6tu16kXT9Po4YMQz_uEd5FN4V3UyAZi6 ► [Selenium framework development] https://www.youtube.com/playlist?list=PL6tu16kXT9Prgk2f37ozqcdZac9pSATf6 ► [Selenium 4] https://www.youtube.com/playlist?list=PL6tu16kXT9PoYFpoLR0OaDuZDjblBPxuO ► [Selenium Grid with Docker] https://www.youtube.com/playlist?list=PL6tu16kXT9PrgjktYZJ9-ISDTEimjjxv7