Local AI for Beginners: GPU vs 16 CPU Cores Explained!

How does local AI work, and what does Ollama actually do? I built the whole process as a seven-button machine and ran a local LLM with Ollama on my own PC: ornith:9b on an AMD Radeon RX 7900 XTX, then on a 16-core Ryzen 9 9950X alone. The graphics card wrote 99.3 tokens per second, the processor 9.3, and one memory number explains the gap. Along the way: how Ollama pulls a model like a Docker image, what a Modelfile is, and the API on localhost:11434 that every tool can use. I tried a new style of video here. Good or Bad? ---- ❤️ SUPPORT THE CHANNEL Patreon: https://www.patreon.com/cw/DIYSmartCode Higgsfield AI — Create AI Videos & Visuals Get 10% OFF here: https://higgsfield.ai?fpr=thomas-dccf7e 🚀 DYNAMOUS AI COMMUNITY Want to learn agentic coding with live daily events and workshops? Check out Dynamous AI: https://dynamous.ai/?code=646a60 Get 10% off here 👉 https://shorturl.smartcode.diy/dynamous_ai_10_percent_discount ⚡ HOSTINGER — RELIABLE HOSTING FOR YOUR PROJECTS (10% OFF) Whether you're shipping a portfolio, a side project, n8n flows, or AI agents — I use Hostinger for fast, affordable VPS + web hosting. Get 10% off here 👉 https://hostinger.com/DIYSMARTCODE n8n: https://n8n.partnerlinks.io/diysmartcode (Affiliate links — costs you nothing, supports the channel.) ---- Chapters 0:00 How Local AI Works: Your GPU Reads, It Doesn't Think 0:35 What Is Ollama? A Local AI Server That Works Like Docker 0:48 Ollama Pull Explained: Manifest, Layers and a 5.6 GB Model File 1:22 Loading a Model Into VRAM: 9.6 GB on a 24 GB RX 7900 XTX 1:52 Tokenizer Explained: 6 Tokens From a 248,320-Token List 2:18 Inside the GPU: 32 Layers, 4,096 Numbers per Token 2:55 How an LLM Picks the Next Word: Token Odds, Temperature and the Modelfile 3:48 Tokens per Second: Memory Bandwidth Sets the Ceiling (960 GB/s) 4:34 GPU vs CPU: Ryzen 9 9950X at 9.3 Tokens per Second (DDR5) 5:07 The Ollama API: localhost:11434 and the OpenAI-Compatible /v1 5:31 What to Buy for Local AI: Memory Speed vs Memory Size What you will see in this local AI breakdown: • What Ollama is: a small server on your PC, and the terminal is just one client • How ollama pull works like docker pull: a manifest, five layers, and the 5.6 GB model layer • Why a 9-billion-parameter model is a 5.6 GB file of numbers, and why 16-bit would be 18 GB • Why Ollama reports 9.6 GB of VRAM for a 5.6 GB model (room for 131,072 tokens of conversation) • How a tokenizer cuts a question into tokens, and why tokens aren't words • What 32 layers and 4,096 numbers per token mean, and why a whole prompt reads in about 50 ms • Real next-token odds logged from the run: hardware 40%, computer 33%, machine 23% • Temperature and the Modelfile: the one inside ornith:9b's download, and how ollama create builds your own model • The tokens-per-second ceiling: memory bandwidth divided by model size (960 GB/s on the GPU, 96 GB/s on dual-channel DDR5) • Measured: 99.3 tok/s on the GPU, 9.3 tok/s on the CPU, and the answer itself changed • The Ollama REST API on port 11434, and the OpenAI-compatible /v1 endpoint for tools built for the cloud • Why Macs with unified memory follow the same rule OLLAMA CHEAT SHEET (every command from this video) ollama pull ornith:9b (download a model: a manifest plus its layers) ollama run ornith:9b (chat in the terminal) ollama ps (what is loaded, and how much VRAM it takes) ollama list (the models on your disk) ollama show --modelfile ornith:9b (the Modelfile inside a download) ollama create my-model -f Modelfile (build your own model from a Modelfile) ollama serve (start the server yourself; it listens on port 11434) curl http://localhost:11434/api/generate -d '{"model": "ornith:9b", "prompt": "How does local AI work?", "stream": false}' OpenAI client: base_url="http://localhost:11434/v1", api_key="ollama" A minimal Modelfile: FROM ornith:9b PARAMETER temperature 0 SYSTEM """Explain it plainly, one step at a time.""" AMD Radeon RX 7900 XTX specs (24 GB GDDR6, 960 GB/s): https://www.amd.com/en/products/graphics/desktops/radeon/7000-series/amd-radeon-rx-7900xtx.html AMD Ryzen 9 9950X specs (16 cores, 2 memory channels): https://www.amd.com/en/products/processors/desktops/ryzen/9000-series/amd-ryzen-9-9950x.html Ollama (the local LLM runner used here): https://ollama.com Ollama Modelfile reference: https://docs.ollama.com/modelfile Ollama API reference: https://docs.ollama.com/api/introduction Ollama OpenAI compatibility (/v1): https://docs.ollama.com/api/openai-compatibility Ornith model on Ollama: https://ollama.com/library/ornith llama.cpp: https://github.com/ggml-org/llama.cpp Would you run AI on your own PC, or stick with ChatGPT? Tell me why in the comments. #LocalAI #LocalLLM #Ollama #OllamaTutorial #WhatIsOllama #OllamaAPI #Modelfile #LLM #RunAILocally #AIHardware #GPU #VRAM #MemoryBandwidth #TokensPerSecond #AMD #RX7900XTX #Ryzen9 #DDR5 #LlamaCpp #PrivateAI #OpenSourceAI #AIExplained #MachineLearning #ArtificialIntelligence #DIYSmartCode