This 2B Voice AI Runs LOCALLY at 158ms — CRAZY Fast, WILD Mistakes (llama.cpp Realtime Test)

I ran LFM3-Omni-2B, a 2B model that hears and speaks, through llama.cpp v0.8.0's new /v1/realtime and /tts on an RTX 4060 laptop: 158 ms to first audio at best, 284 ms on average, 3.3 GB of VRAM, no cloud. It also made some funny mistakes. I tested ten spoken tasks, interruptions, English vs Hindi, background noise, and a Whisper + LLM + Kokoro baseline. All numbers are from my own run (10 prompts, Windows TTS voices, one run each, not a benchmark). 0:00 158 ms to first audio 0:37 What shipped in llama.cpp v0.8.0 1:06 Setup on an RTX 4060 laptop 1:30 Test 1: listening (English) 2:01 Test 2: talking back, latency 2:40 What it got wrong 3:20 Interrupting it 3:35 Hindi and Hinglish 4:17 Vs Whisper + LLM + Kokoro 4:47 /tts and the verdict RunPod (cheap, reliable GPUs): https://get.runpod.io/pe48 Release: https://github.com/ggml-org/llama.cpp/releases/tag/v0.8.0 Model: https://huggingface.co/LiquidAI/LFM3-Omni-2B-GGUF Not tested: tool calling, live microphone interruption. Thumbnail and some visuals are AI-generated. Get GPUs Runpod: https://get.runpod.io/pe48 Get Hostinger: https://www.hostg.xyz/SHJiu Coupon Code "PROMPT" 10% discount 🔗 Connect with me: Email → [email protected] Ko-fi → https://ko-fi.com/promptengineer Patreon → https://www.patreon.com/PromptEngineer975 Book a Call → https://calendly.com/prompt-engineer48/call GitHub → https://github.com/PromptEngineer48 Twitter/X → https://twitter.com/prompt48 Website → https://promptengineer48.com/