200 Tokens a Second on a M5 Max? Where It's Real, Where It Isn't

Same M5 Max, same model file, same answer, up to seven times faster. In my last video gpt-oss-120b ran at 79.2 tokens a second on this Mac, and I keep hearing about 200. So I measured every method I could find that makes the same hardware faster, baseline first, including the ones that didn't help. The short version: three levers. The engine (llama.cpp vs MLX) is within about 10% for one dense model. Speculative decoding (a drafter guesses, the big model checks, greedy output stays identical) is the big one: Qwen3.8-27B went from 23.9 to 132 tok/s on code with TensorFold and DFlash2, and from 22.2 to 157 on a file edit. Prompt caching doesn't speed up writing but cuts the wait before the first word from ~25 s to under half a second on a 13k-token file. And the 200 tok/s figures are real, on a small-active-parameter model writing predictable text for one person. For gpt-oss-120b, last video's model, a newer llama.cpp alone was a draw for writing (79 → 84 tok/s on the same test) and its EAGLE-3 drafter didn't pay on this Mac. Chapters 0:00 Intro 0:54 The ceiling 1:38 Lever 1: the engine 2:38 Lever 2: speculative decoding 3:37 Drafting, measured 4:25 Code vs prose vs edits 5:17 The n-gram trap 6:07 Where 200 tok/s comes from 8:02 Lever 3: prompt cache 9:17 What didn't help 10:09 Other hardware 11:08 Recipe Measured on my M5 Max (128 GB, 19 Oct 2026; greedy, 512 tokens, thinking off, median of 5 runs, decode tok/s with prefill timed separately) Qwen3.8-27B 4-bit, code prompt / prose prompt: - llama.cpp b11724, plain: 23.9 / 20.6 - mlx-lm 0.32.2, plain: 22.0 / 23.0 - llama.cpp + MTP heads: 35.4 / 25.8 - llama.cpp + DFlash2 drafter: 53.8 / 28.0 - LM Studio 0.4.26 Splash + DFlash2: 85.6 / 37.3 - TensorFold v1.1.0 + DFlash2: 132.3 / 54.3 (drafting off: 19.7 / 20.9) - MTPLX 2.12.0, MTP on / off: 63.7 / 38.2 vs 20.4 / 15.9 File edit (rename a class + docstrings, 721-token prompt): mlx-lm 22.2, llama.cpp 18.7, llama.cpp n-gram 27.2, llama.cpp DFlash2 65.0, TensorFold DFlash2 157.3 n-gram on code: 174.9 only when the same question was repeated on one server; 17.6–18.4 on a fresh one Nemotron 3.5 Lightning (30B MoE, ~3B active), code / prose: mlx-lm 129.8 / 121.9; TensorFold drafting off 154.7 / 155.2; TensorFold MTP drafting 257.4 / 200.0 gpt-oss-120b MXFP4, same llama-bench test as last video (pp512 / tg128, median of 9 samples): b10229 1,302 / 79.0, b11724 2,015 / 84.3 (writing +7%, within the run-to-run spread; prompt reading +55%) gpt-oss-120b on llama.cpp b11724, code / prose: plain first run 96.1 / 83.3, plain warm rerun 80.3 / 76.9, EAGLE-3 81.2 / 69.3 (drafts accepted 58% / 46%), EAGLE-3 with 2-token drafts 72.4 / 68.5 Prompt cache, ~13k-token file then a second question (first word after): llama.cpp 23 s → 1.3 s; mlx-lm 25 s → 0.4 s and TensorFold 28 s → 0.1 s with the file in its own message Load average 4–20 during runs (other work on the machine); treat differences under ~10% as noise. Sources (checked 19 October 2026; vendor and project figures are self-reported) - llama.cpp release b11724: https://github.com/ggml-org/llama.cpp/releases/tag/b11724 - llama.cpp speculative decoding docs: https://github.com/ggml-org/llama.cpp/blob/master/docs/speculative.md - TensorFold: https://github.com/ashhart/TensorFold - TensorFold Zig preview figures (M5 Max, Nemotron 3.5 Lightning): https://github.com/ashhart/TensorFold/blob/main/ZIG-PREVIEW.md - TensorFold Qwen3.8-27B recipe (M5 Ultra and DGX Spark figures): https://github.com/ashhart/TensorFold/blob/main/docs/recipes/qwen3.8-27b.md - MTPLX: https://github.com/Youssofal/MTPLX - oMLX: https://github.com/jundot/omlx - mlx-lm: https://github.com/ml-explore/mlx-lm - Strix Halo llama.cpp tests (visorcraft): https://github.com/visorcraft/strix-halo-llm-perf - Jarvislabs, Gemma 4 MTP vs DFlash on one H100 (12 May 2026): https://jarvislabs.ai/blog/gemma-4-mtp-vs-dflash-benchmark - NVIDIA DGX Spark speculative decoding playbook: https://build.nvidia.com/spark/speculative-decoding My related videos - The 128 GB local-AI buyer's guide (previous video): https://youtu.be/gYZC7p9ZkJw - Can an M5 Max code locally?: https://youtu.be/TjhFzscnad4 - Water-cooled M5 Max: https://youtu.be/tj8z9KmWXmg - Ryzen AI Max+ PRO 495 explained: https://youtu.be/fzI1oZ5fe6Q No footage from other creators is used. Numbers from hardware I don't own are published figures and are credited on screen.