Connect Two NVIDIA DGX Sparks Together to Run Large Models
In this video, I'm taking a deep dive into connecting two NVIDIA DGX Spark-style devices using a ConnectX-7 port and a QSFP112 cable. I'll break down the hardware layout (mixing a Gigabyte and ASUS G10 variants), explain the PCIe lane limitations of the GB10 chip that require interface bonding, and explore how RDMA (Remote Direct Memory Access) and RoCE v2 (RDMA over Converged Ethernet) bypass the OS kernel for ultra-low latency. Finally, I'll walk through the implementation using NVIDIA’s playbooks, configure Netplan, NCCL (NVIDIA Collective Communication Library), and deploy vLLM via Docker. Are you running a multi-node local LLM cluster? Let me know your setup and experiences in the comments below! 🔔 **Subscribe** for more hands‑on cloud tutorials! *To follow along, check out my GitHub page! All the commands and instructions from this video are in a README file:* *https://github.com/heavy-metal-cloud/youtube/blob/main/videos/bare-metal-ai/04-dgx-spark-running-ai-two/README.md* 00:00:27 Hardware Overview: Gigabyte & ASUS G10 Variants 00:00:45 The QSFP112 Cable Architecture Explained 00:02:20 The GB10 Chip: Why the ConnectX-7 Ports Are Bonded 00:03:41 RDMA vs. TCP/IP for High-Speed Data Centers 00:04:57 Step 1: Initial NVIDIA Playbook Setup & Netplan Config 00:06:37 Step 2: What is NCCL (Nvidia Collective Communication Library)? 00:07:14 Step 3: Setting Up vLLM and Docker 00:07:59 Best Practice for Downloading Hugging Face Models Multi-Node 00:08:32 Benchmark 1: MiniMax 2.7 (230B Parameter Model) 00:09:07 Benchmark 2: Nemotron-3 Super (Tokens Per Second Boost) 00:09:37 Tensor Parallel Size vs. Pipeline Parallel Size in vLLM Links referenced in this video: https://build.nvidia.com/spark/connect-two-sparks/stacked-sparks https://build.nvidia.com/spark/nccl https://www.fs.com/blog/a-quick-look-at-the-differences-rdma-vs-tcpip-2945.html https://www.fs.com/glossary/remote-direct-memory-access-rdma-g105.html https://www.servethehome.com/the-nvidia-gb10-connectx-7-200gbe-networking-is-really-different/ https://github.com/eugr/spark-vllm-docker https://vllm.ai/blog/2026-06-01-vllm-dgx-spark https://developer.nvidia.com/nccl https://docs.ray.io/en/latest/ray-overview/getting-started.html Amazon Affiliate Links https://amzn.to/3QDGmgJ https://amzn.to/4aUXKnW