New Local AI Inference Engine Makes Qwen3.8-27B 2× Faster on Apple Silicon 🚀
In this video, I explore Splash, an open-source inference engine from Inco AI designed specifically for high-performance local AI inference on Mac. Instead of being a general-purpose runtime for hundreds of models, Splash optimizes the inference stack around individual models using custom Metal kernels, DFlash 2 speculative decoding, model-aware memory management, and prefix-aware caching. With Qwen3.8-27B, Inco reports around 2× faster decode performance compared with the next-fastest engine in its published short-prompt benchmark — 74 tok/s versus 38 tok/s on an M5 Pro. Chapters: 00:00 - Introduction 04:00 - Running QWEN3.8-27B without MTP 05:13 - Running QWEN3.8-27B with MTP enabled 07:55 - Running QWEN3.8-27B with Splash Engine My AI related Courses for you to empower today: Learn my latest course launched this week on ZERO Code AI Tests: Automate Software Tests in Plain English: https://www.udemy.com/course/zero-code-ai-testing/?couponCode=LEARN_SEP_26 Use code: LEARN_SEP_26 for 85% OFF (limited time) My Other AI related Courses for you to empower today: To Learn more about building your own AI Agents and Testing AI Agents, follow the courses below available in Udemy with Discount coupon codes. Use coupon code : LEARN_SEP_26 🤖 AI-Driven Test Automation: Playwright, Selenium, LLMs & More https://www.udemy.com/course/ai-test-automation-selenium-playwright-llms/?couponCode=LEARN_SEP_26 🤖 https://www.udemy.com/course/genai-multi-agent-software-testing/?couponCode=LEARN_SEP_26 🤖Test AI & LLM App with DeepEval, RAGAs & more using Ollama https://www.udemy.com/course/ai-testing-deepeval-ragas-ollama/?couponCode=LEARN_SEP_26 🤖Build & Test AI Agents, ChatBot, RAG with Ollama & Local LLM https://www.udemy.com/course/build-ai-agent-chatbot-rag-langchain-local-llm/?couponCode=LEARN_SEP_26 🤖2025 - Understand ,Test ,Fine-tune AI Model with HuggingFace https://www.udemy.com/course/ai-with-huggingface/?couponCode=LEARN_SEP_26 🤖2025 - Using Generative AI in Software Automation Testing https://www.udemy.com/course/generative-ai-in-software-automation-testing/?couponCode=LEARN_SEP_26 🤖AgentQL: https://youtu.be/XzmkFSqTgtM 🌐 Local Deepseek R1 for BrowserUse - https://youtu.be/NvClPPI4sK0 🌐 Browser Use - https://youtu.be/O1GzsqJFpuI ➜ Discount coupon codes for all the courses: LEARN_SEP_26 ► [Advanced Framework development Course in Selenium C#] https://www.udemy.com/course/framework-development-with-selenium-csharp-advanced/ ► [Advanced Framework development Course in Playwright C#] https://www.udemy.com/course/framework-development-with-playwright-dotnet/ #LocalAI #Qwen #AppleSilicon #M5Max #Splash #LLM #AI #ExecuteAutomation ► [Udemy] https://www.udemy.com/user/karthik-kk/ ► [XUnit with Selenium] https://www.youtube.com/playlist?list=PL6tu16kXT9Prbp7hTEb-M-luddsRJZKJ2 ► [Git Basics] https://www.youtube.com/playlist?list=PL6tu16kXT9PqIcfMOE9hX81cVOnmiM1E2 ► [SpringBoot for Testing] https://www.youtube.com/playlist?list=PL6tu16kXT9PrDr6kMGQ-CgnvCsFxrq1eS Selenium and C# ► [C# for automation testing] https://www.youtube.com/playlist?list=PL6tu16kXT9Pp3NFZgLbPZXEykeGQwxGSx ► [Selenium with C#] https://www.youtube.com/watch?v=mluLgBywW0Y&list=PL6tu16kXT9PqKSouJUV6sRVgmcKs-VCqo ► [BDD with Specflow] https://www.youtube.com/playlist?list=PL6tu16kXT9Pp3wrsaYyNRnK1QkvVv6qdI ► [BDD with Selenium] https://www.youtube.com/playlist?list=PL6tu16kXT9PrAjLHMTtttMybnV1kDpCXk ► [Selenium .NET Core] https://www.youtube.com/playlist?list=PL6tu16kXT9PqLJfuuLcvHnE61pzD8Dq9M Selenium &Java ► [Cucumber with Selenium] https://www.youtube.com/playlist?list=PL6tu16kXT9PpteusHGISu_lHcV6MbBtA6 ► [Cucumber with Selenium] https://www.youtube.com/playlist?list=PL6tu16kXT9Pqr70SZlwcmTSAfOw_0Qj3R ► [Cucumber 4 Upgrade] https://www.youtube.com/playlist?list=PL6tu16kXT9PoT4gRr5F00r1lPaTszk_-q ► [Selenium Grid] https://www.youtube.com/playlist?list=PL6tu16kXT9Po4YMQz_uEd5FN4V3UyAZi6 ► [Selenium framework development] https://www.youtube.com/playlist?list=PL6tu16kXT9Prgk2f37ozqcdZac9pSATf6 ► [Selenium 4] https://www.youtube.com/playlist?list=PL6tu16kXT9PoYFpoLR0OaDuZDjblBPxuO ► [Selenium Grid with Docker] https://www.youtube.com/playlist?list=PL6tu16kXT9PrgjktYZJ9-ISDTEimjjxv7