More on that SBC for LLMs

This second video gives you the real story on what Luckfox Omni3576 — a Jetson-like SBC powered by the RK3576 SoC — can actually do. I benchmark its performance to compare it against the Jetson Orin Nano and Raspberry Pi 5 using Gemma 2B, Llama 3.1 8B, and Qwen2-VL-2B models, run a few more examples, including multi-modal inference and also measure the power consumption. #aliexpress links: https://s.click.aliexpress.com/e/_EwnUdss Luckfox Omni3576 board https://s.click.aliexpress.com/e/_ExaOYU8 Luckfox Omni3576 module https://s.click.aliexpress.com/e/_ExScjyg Raspberry Pi 5 https://s.click.aliexpress.com/e/_oFVbErR Nvidia Jetson Orin Nano