What Building an AI Model Actually Looks Like | Pradeep Natarajan of Amazon AGI

Can AI be both accurate and creative? Pradeep Natarajan, Senior Principal Scientist at Amazon AGI joins Peter Gostev on Arena Conversations to talk about what building an AI model actually looks like day to day: the data work, the training recipe, the evals, and why the metrics we choose shape what models become. - Why Pradeep estimates 90-95% of gains come from getting the fundamentals right, not from a single algorithmic breakthrough - How evals, not just compute, become the real differentiator - Why harness optimization is as important as the model itself - How the metrics we define (like accuracy and style) steer models away from creativity 0:00 What model building looks like day to day 3:33 Compute vs. data as the bottleneck 5:15 The core training recipe: pre-training, SFT, and RLHF 7:15 Big breakthroughs vs. accumulated small insights (PPO to GRPO) 8:28 Could you build a frontier model just from the papers? 11:00 Do models really generalize? 12:47 Why evals are the real moat 16:12 The unsolved problem: evaluating models in real deployments 18:10 Why harnesses matter as much as the model 21:20 How every AI leap opens up new fields 22:25 Recursive self-improvement and human-machine co-intelligence 26:12 From magical to mundane: how fast expectations reset 28:46 Can models do real research? 32:26 Why AI writing feels like it's getting worse 32:59 Metrics vs. creativity: the "Mount Everest" problem 35:36 Will AI writing get better? #arenaai #amazonai #llm