NPU on Radxa Dragon Q6A for Local LLMs

Hardware.ai
3,649 views •

I spent a full week debugging the Radxa Dragon Q6A so you don't have to. My eyes are literally "Linux red" after fighting with drivers and version mismatches, but I finally got the NPU working for local LLMs. The NPU doesn't make things faster, rather it makes inference more efficient and frees the CPU for other tasks. In this video, I'm sharing the exact step-by-step process that actually worked - and more importantly, the things that definitely DID NOT work (like llama.cpp with Hexagon on the 6490). Github gist: WIP

Keyboard shortcuts

On. Switch them off if they collide with your assistive tools; ? still opens this sheet.

Go to

Press g then the letter.

  • gh Latest
  • gs Sources
  • gm Media
  • gv Videos
  • gp Podcasts
  • gc Calendar
  • gd Decoder
  • gz Dataviz
  • ga Datasets
  • gb Blog
  • gk Markets
  • gj Careers
  • gn Prompt Notebook

On this page

  • / Focus search, where there is one
  • t Back to top
  • ? This list
  • Esc Close