Google's Self-Improving AI Agent

Google Cloud AI Research just open-sourced Harness², a recursive framework that lets AI agents automatically upgrade their own surrounding systems—prompts, tools, and memory—without changing a single model weight. On benchmarks, a single self-refinement boosted an open model from 59.6% to 71.4%, allowing weaker models to beat stronger ones simply by fixing their harnesses. In this breakdown, we look at how the Propose-Probe-Compose loop works, how agents evaluate themselves without an answer key, and what this extra compute costs you. Key Takeaways: • Upgrades Without Weights: How GLM-5 jumped from 59.6% to 71.4% on the LAB benchmark with zero model retraining. • The Propose-Probe-Compose Loop: Generating general and task-specific harness edits, then probing behavioral differences against a reference run. • No Answer Key Needed: Evaluating novel, open-world tasks by observing behavioral contrasts rather than checking against pre-written ground truth. • Persistent Memory: Storing adopted harness patches and lessons into an EXPERIENCE.md file for future tasks to reuse. • Weaker Beats Stronger: How an upgraded GLM-5 outperformed base Gemini 3.5 Flash on WorkBuddy-Bench (75.7 vs 72.4). • The Compute Cost: Running roughly four rollouts per task (reference, two candidates, merge) plus calls to the editing model. 🔗 Sources & Documentation: • Harness² Research Paper (arXiv 2610.14224): https://arxiv.org/abs/2610.14224 • Google Research Harness² GitHub Repository: https://github.com/google-research/harness2/ • Wavect Technical Review on Harness² Cost and Safety: https://wavect.io/blog/google-harness2-setup-cost-safety/ 🔔 Subscribe: https://www.youtube.com/channel/UC0DZj1PNa_Fp0MU6uPSKv5w?sub_confirmation=1 💙 Become a Member: https://www.youtube.com/channel/UC0DZj1PNa_Fp0MU6uPSKv5w/join 🐦 Twitter/X: https://x.com/cloud_codes 💬 Discord: https://discord.gg/mhgXXmhrh #google #ai #aiagents #opensourceai #machinelearning #llm #harness #cloudcodes