ADAS and Autonomous Vehicle Data Curation with MCAP | Voxel51

Curate ADAS and autonomous vehicle data in MCAP with FiftyOne. Murilo Gustineli, ML engineer at Voxel51, shows synced LiDAR and camera playback, embeddings, and temporal events. He starts with Berkeley DeepDrive (BDD) images, using embeddings to surface rainy frames labeled "daytime" and tunnel images that were never labeled. Then he moves to MCAP: front, left, and back cameras played in sync with LiDAR, semantic map and ego path overlays, and live odometry plots. In Voxel51, he compares stream embeddings from Qwen3-VL 8B and 2B (the 2B clusters are less clean) and finds similar moments across drives. He also builds derived events like "car braking near pedestrians" from acceleration data and VLM detections. One "swerving" hit turns out to be a left turn, a false positive he calls out. The Q&A ends with the agentic labeler detecting road signs. ⏱️ Chapters 0:00 Intro 1:13 Curating BDD driving images 3:38 How the embeddings panel works 4:58 Finding label errors with embeddings 6:42 Saved views for your team 7:35 What MCAP is 8:16 Synced camera, LiDAR, and map playback 11:14 Layouts and odometry plots 12:37 Stream embeddings: Qwen3-VL 8B vs. 2B 15:17 3D embeddings and temporal similarity 17:37 Find similar streams 18:31 Events from sensor heuristics 22:26 Construction and swerving events 24:47 Annotating captured events 26:03 Q&A: camera metadata 28:29 Q&A: detecting road signs with the agentic labeler 🔗 Learn more 10 MCAP datasets for physical AI: https://voxel51.com/blog/mcap-datasets-multimodal-robotics-fiftyone Self-driving dataset guide: https://docs.voxel51.com/getting_started/self_driving/index.html FiftyOne Multimodal docs: https://docs.voxel51.com/user_guide/multimodal.html Agentic Labeling docs: https://docs.voxel51.com/enterprise/agentic_labeling.html Voxel51 for autonomous vehicles: https://voxel51.com/industries/autonomous-vehicles-systems 💬 Join the community: https://community.voxel51.com FiftyOne is the open source multimodal data platform for physical AI. Voxel51 is the enterprise platform built on it, adding annotation workflows, auto labeling, GPU-backed models, and team collaboration. Together they help ML teams curate, annotate, and evaluate data for visual AI and physical AI models, across images, video, 3D, and multimodal sensor data. #FiftyOne #PhysicalAI #AutonomousVehicles