GPT-6 Astra vs. Gemini, Qwen, Claude | We Tested Them on 4,000 Images

Which vision-language model (VLM) is best at object detection? Here are the results from benchmarking GPT-6 Astra, Gemini, Claude, and Qwen on over 4,000 from 20 domains. In this video, Matvei Popov, a research engineer at Roboflow, introduces Roboflow 100-VL (RF100-VL), a computer vision benchmark of real-world datasets. He shares the results of zero-shot and few-shot performance of frontier models on 20 datasets from the benchmark. You'll see how the models were prompted, which models improved the most with few-shot tests, and an overall comparison of which models did best on which datasets. Chapters 0:00 Preview: How Far VLM Object Detection Has Come 1:16 What Is Roboflow 100-VL (RF100-VL)? 4:01 Few-Shot Examples and Annotator Instructions 5:20 RF20-VL-FSOD: 20 Datasets Across 7 Categories 7:24 Zero-Shot vs One-Shot vs 10-Shot Prompting 9:26 mAP50-95 vs Rex F1@mIoU: Scoring VLM Detections 11:52 How to Prompt a VLM for Bounding Boxes 13:07 Few-Shot Prompting with Alternating Conversation Turns 14:22 Results: Four VLMs on RF20-VL-FSOD 16:01 Few-Shot Examples Now Improve VLM Detection 16:55 GPT-6 Astra vs Gemini 3.8 Flash vs Qwen3.8-Max vs Claude Fable 5.1 20:00 Ambiguous Class Names: When Few-Shot Examples Help Most 22:41 Which VLM to Use for Your Industry 24:28 Examples: When Few-Shot References Help and When They Hurt 26:58 Key Takeaways = Resources = RF100-VL benchmark: https://rf100-vl.org RF100-VL paper: https://arxiv.org/abs/2505.20612 RF100-VL code and datasets: https://github.com/roboflow/rf100-vl Compare vision models on your own images: https://playground.roboflow.com Join a future live session: https://roboflow.com/webinar